Large scale blocking of TLS-based censorship circumvention tools in China #129
Comments
中国大规模地封锁基于TLS的翻墙服务器自北京时间2022年10月3日起，超过一百名用户报告他们至少有一台基于TLS的翻墙服务器被封锁了。被封锁的服务器使用的协议包括了trojan，Xray，V2Ray TLS+Websocket，VLESS，以及gRPC。我们还未收到任何naiveproxy被封锁的消息。 下面是我们总结的关于这次封锁的一些信息，以其我们的一些推测和分析 封锁先是针对翻墙服务的端口。如果用户在端口被封后，改换了端口，那么整个服务器都会被封锁。需要指出，封锁似乎只是基于端口或IP地址，与翻墙服务有关的域名似乎并没有被加入到GFW的DNS或SNI黑名单中。 尽管大多数用户报告443端口被封，一部分使用非443端口的用户也报告了封锁。尽管大多数用户的服务器在流行的VPS提供商那里（比如），但至少有一位用户位于欧洲的家中的服务器也被封锁了。 在一些案例中（并非全部案例中），封锁是动态的：用户通过浏览器还是可以直接访问翻墙端口，但同一个端口，用翻墙软件就连不通。 所有以上的信息都指向GFW已经可以精准的识别并封锁这些翻墙协议，而并非简单地封锁所有的443端口，或封锁所有的流行机房。 基于以上信息，我们推测（但还未进行实证性的测量），这些封锁可能与翻墙软件客户端发出的Clienthello指纹相关。开发者们或许可以考虑采用uTLS。这个论文阅读小组，这篇总结，以及这篇博文都是关于TLS指纹的，也许会有帮助。 下一步，我们将调查GFW是否真的使用了客户端发出的TLS指纹来识别这些协议。与此同时，如果您有任何翻墙服务器被封锁，或者有任何可以证实或反驳我们的推测的例子，我们都欢迎您或公开地或私下地与我们分享。因为这会帮助我们快速定位许多问题的根源。我们私下的联系方式可见GFW Report的页脚。 |
|
一直没用443也同样被封，但偶尔是还是可以访问（卡，有中断现象） I never used port 443, my non-443 port still got blocked, but sometimes I can still access it (lagging, sometimes with interruptions) |
|
最近梯子间歇性中断，所幸没有被gfw直接ban掉 The ladder [circumvention tool] gets disrupted intermittently, but fortunately the server is not banned by the GFW. |
|
是否可以基于奥卡姆剃刀原则认为目前GFW的行为模式已经进化到了「能够识别基于TLS的翻墙协议」或者「不管什么协议只要出现长时间大流量多IP连接的境外IP就Ban」呢？ Is it possible to argue that the current behavior pattern of GFW has evolved based on Occam's Razor to the point of "being able to identify TLS-based wall protocols" or "Ban any protocol as long as there is a long period of heavy traffic with multiple IP connections from outside the country"? |
|
确实从昨天开始我使用的机场频繁掉线需要重连 It's true that since yesterday the airport I use has been dropping connections frequently and needs to be reconnected |
|
nginx前置的服务器一样被封了，就足够说明tls指纹不是关键。也别尬吹什么navie了，追求稳定不如中转CDN，追求速度不如hy。另外一提，用tuic和hy的vps暂未见中枪。 My server has ngnix as a frontend application but it still got blocked, which is enough to show that TLS fingerprints are not the key. Do not embarrassingly boast about naiveproxy. If you need stability, you'd better use a CDN to relay the traffic; if you need speed, you'd better use hy. BTW, VPSes that uses tuic and hy have not been observed to be banned. |
TLS 指纹识别的是客户端，即 TLS 会话是否是由翻墙软件作为客户端发起的，而非看连接到的是什么服务器。
TLS fingerprinting identifies the client, i.e. whether the TLS session was initiated by the wall-jumping software as a client, rather than looking at what server is connected to. |
|
我的翻墙服务器（testserver1.mckuhei.ml）用的trojan + fallback 443端口被封禁，但是浏览器同时也无法访问443端口 My wall server (testserver1.mckuhei.ml) with trojan + fallback port 443 is blocked, but the browser can't access port 443 at the same time |
使用ssh时，如果同时域名加密，稳定性大大提高。
When using ssh, if the domain name is encrypted at the same time, the stability is greatly improved. |
|
Very interesting finding, thanks for sharing. If naiveproxy is indeed not blocked, it would be consistent with the theory that it may be TLS fingerprinting, as it uses the Chrome stack and doesn't have a distinguishable fingerprint. |
我前置nginx stream分流trojan回落到9100端口，node_exporter的端口，但是只能http+443在浏览器访问到9100内容，https则是ERR_CONNECTION_CLOSED（因为前置分流的nginx没有监听443 ssl,使用了sni_preread缘故），担心迟早被封，能否让回落的http网站upgrade到https连接 I front nginx stream shunt for trojan fallback to port 9100--node_exporter port, but only http + 443 in the browser can access the 9100 content, https is ERR_CONNECTION_CLOSED (because the front shunt nginx does not listen ssl protocol for port 443, cause using the sni_preread reason), worried about sooner or later blocked, how to fallback http site upgrade to https connection |
|
是否可以关注一下都是那些数据中心和供应商被识别，理论上大规模过滤TLS/HTTPS不太现实。 BTW： Is it possible to focus on all those data centers and providers are identified, theoretically mass filtering TLS/HTTPS is not very realistic. BTW: |
|
可能与端口无关，我的机场端口号都在20000以上，有大约三分之一的节点挂了，没挂的也全部都出现高丢包率现象了 It may be unrelated to port number: the ports used by my airports are all above 20000. Approximately one third of them got banned, and the ones that were not banned experienced high rates of packet drops. |
不知道，我是装了个apache，然后启用了h2c模块，然后再让443fallback到80就好了
I don't know, I installed an apache, then enabled the h2c module, and then let 443 fallback to 80 on it |
|
Greetings, Lumière Élevé here. |
Is the IP address of the blocked server outside of China? At the same time, I was thinking that if a foreign server has been maintaining a high traffic, long time connection with a single Chinese IP, and no long time, high traffic transmission with other Chinese IPs, this feature is likely to have been learned by GFW and applied to blocking, ignoring any protocol, encryption |
|
There is evidence both for and against the hypothesis of TLS fingerprinting in this thread. For:
Against:
Of course, there can be more than one thing happening. It may be simultaneously true that connections are being blocked by TLS fingerprint, and that there is some kind of flow-based access pattern analysis to identify TLS proxies.
There is some research on detecting proxies based on common access patterns by clients. This research paper from 2018 uses spectral clustering of an access pattern graph in an attempt to distinguish proxy users from non–proxy users. See Sections 3.2 and 3.4.B. The target of the paper is web proxies, but the technique could be applied to other kinds of proxies. 一种基于多维特征分析的网页代理服务发现方法 The paper gives a rationale for clustering access patterns:
I am not claiming that something like this is happening in this case, only saying that there is some evidence of it being researched. |
|
我有4台vps，只有一台常用的被封了443端口，之后又在这台常用的vps上架设了tuic，6小时内被封8443端口，只看了youtube视频 I have 4 vps, only one commonly used. one was blocked on port 443, after that and setting up tuic on this commonly used vps, within 6 hours port 8443 was blocked, only watched youtube videos |
|
During a sudden change like this, circumvention tool developers have an advantage over the GFW: agility: In the short term, you can change the protocol faster than the GFW can react. The GFW has, no doubt, been planning today's blocking of TLS-based circumvention tools for weeks or months; it will likely not be able to do another such large-scale action without more planning and coordination. By acting quickly, you can perhaps keep the advantage during this time of intensified blocking. We already have the first necessary element, which is quick awareness of the problem. Now we can try countermeasures, for example tweaking the TLS fingerprint as @gfw-report suggested. |
|
According to the paper, the restriction is applied based on the clients(or the users) themselves' common features. |
|
For the realistic problems, at least everyone can use a large public CDN server to mix the data amongest the others' and using some tricks of Firewall to prevent GFW from active detecting. (Like using a specific UA, for a instance, change "Chrome" to "Chremo" and filtering based on the UA) |
If I understand the paper correctly, the shared features among users are only their access patterns. Two users are similar if they access similar destinations. However there is another component to the system, which is is crawling and extracting features from web pages, such as whether the URL has a certain structure, whether there is a |
|
我个人使用的vps，vmess+ws+tls。已经正常运行2年了，之前都没有被封过。但最近两天443端口被封了两次，表现为封一会后解封，几小时又再次被封。只有443端口被封，80和其他端口都是正常连接的。 My personal commonly used vps, using vmess+ws+tls. It has been running normally for 2 years and has not been blocked before. However, in the past two days, port 443 has been blocked twice. It is shown that it was blocked for a while and then unblocked, and then blocked again within a few hours. Only 443 ports are blocked, 80 and other ports are connected normally. |
|
10台vps用的vmess+tls+web，伪装站用的网盘，目前毫发无伤。 10 VPSes with vmess+tls+web, disguised station with the web site, currently unharmed. |
|
After reading source code of v2ray, it seems TLS fingerprint feature was a experimental feature in 2019, but it was then removed for unknown reason. Xray have at least implemented uTLS, but it was not enabled by default. I guess that is the reason why v2ray based client was banned. |
|
备案的可以直接查，搭代理的总是不敢备案的吧。 |
|
SS-AEAD、Trojan 存活情况要比 VMESS+ANY+TLS 好的多，统计了十几台各个国家地区的服务器，VMESS 这边仅剩几个北美的服务器存活，而 SS-AEAD、Trojan 基本上都存活下来了。 不只是 443 被封锁，使用非标准端口，一样被封锁。443 不是神，在 GFW 面前任何端口都没有区别。 我在 10 月 2 日就观察到了被封锁现象，不只是 10 月 3 日，但是大规模封锁是在 10 月 3 日中午。10 月 2 日的封禁方式是封锁该 IP 的全部协议，10 月 3 日则大多表现为只封禁端口，而不封锁整个IP。在 10 月 2 日后侥幸存活下来的服务器，大多都采用封禁端口的形式。即使更换端口，虽然短时间可以连通，但是会在几个小时内，端口继续被封禁。 而将这些服务器转而使用 VMESS+ANY，即不加 TLS，则可以正常使用不会被封禁。自 10 月 3 日至今，这些不使用 TLS 的 VMESS 服务器仍然存活。也有部分服务器使用了 Trojan，目前也是存活完好。 有理由怀疑，VMESS 的 TLS 在实现方式上是不是有特征可供利用。 完全解释不通 Trojan TLS 基本不会被封锁的现象，均使用了相同的 v2ray 核心，以及大多数人使用 Clash 核心作为客户端。 截至到 22-10-16 这些 Trojan 服务器也被封锁，即使添加回落等方式。 SS-AEAD and Trojan survive much better than VMESS+ANY+TLS. After counting a dozen servers from various countries, only a few North American servers survive on the VMESS side, while SS-AEAD and Trojan basically survive. Not only 443 is blocked, but also when using non-standard ports. 443 is not a god, and no port makes any difference in front of GFW. I observed the blocking phenomenon on October 2, not just October 3, but at noon on October 3. The blocking method on October 2 was to block all protocols of the IP, while on October 3 it was mostly to block the port but not the whole IP. Even if the ports were changed, they could be connected for a short time, but within a few hours, the ports would continue to be blocked. If these servers are switched to VMESS+ANY, i.e. without TLS, they can be used normally without being blocked. Since October 3, these VMESS servers without TLS are still alive. There are also some servers that use Trojan, but they are still alive and well. It is reasonable to wonder if there are features in VMESS's TLS implementation that could be exploited. It does not make sense at all that Trojan TLS would be largely unblocked, all use the same v2ray core, and most use the Clash core as a client. As of 22-10-16, these Trojan servers have also been blocked. Even adding fallbacks etc. |
|
I have removed the CDN from my direct connection to my server as a test. I will run for a week to see. I am running v2ray vmess+ws+TLS with uTLS enabled. I am the only one using this server, so data draw shouldn't be higher than 1-2TB a month. I will provide feedback after a week if any blocking. |
|
There are someone directly connecting during the period with good luckiness that without blocked.
(but don't know how you configure client, some subtle configuration may cause some difference.)
Mar 17, 2023 16:57:23 Heclalava ***@***.***>:
…
I have removed the CDN from my direct connection to my server as a test. I will run for a week to see. I am running v2ray vmess+ws+TLS with uTLS enabled. I am the only one using this server, so data draw shouldn't be higher than 1-2TB a month.
I will provide feedback after a week if any blocking.
—
Reply to this email directly, view it on GitHub[#129 (comment)], or unsubscribe[https://github.com/notifications/unsubscribe-auth/AKGBAYBCGFTG4C37H3CQUOTW4QRPFANCNFSM6AAAAAAQ4LAPQ4].
You are receiving this because you were mentioned.[Tracking image][https://github.com/notifications/beacon/AKGBAYFUT2HDCIOLRGLFUUDW4QRPFA5CNFSM6AAAAAAQ4LAPQ6WGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTSX2LUXS.gif]
|
|
(download Meta llama models recently, which costs more than 210 GB traffic. And all things normal. תודה לאל)
Mar 17, 2023 22:40:12 Nanyu ***@***.***>:
… There are someone directly connecting during the period with good luckiness that without blocked.
(but don't know how you configure client, some subtle configuration may cause some difference.)
Mar 17, 2023 16:57:23 Heclalava ***@***.***>:
>
>
> I have removed the CDN from my direct connection to my server as a test. I will run for a week to see. I am running v2ray vmess+ws+TLS with uTLS enabled. I am the only one using this server, so data draw shouldn't be higher than 1-2TB a month.
>
> I will provide feedback after a week if any blocking.
>
> —
> Reply to this email directly, view it on GitHub[#129 (comment)], or unsubscribe[https://github.com/notifications/unsubscribe-auth/AKGBAYBCGFTG4C37H3CQUOTW4QRPFANCNFSM6AAAAAAQ4LAPQ4].
> You are receiving this because you were mentioned.[Tracking image][https://github.com/notifications/beacon/AKGBAYFUT2HDCIOLRGLFUUDW4QRPFA5CNFSM6AAAAAAQ4LAPQ6WGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTSX2LUXS.gif]
>
—
Reply to this email directly, view it on GitHub[#129 (comment)], or unsubscribe[https://github.com/notifications/unsubscribe-auth/AKGBAYCOX667D3ML4MPV34DW4RZUZANCNFSM6AAAAAAQ4LAPQ4].
You are receiving this because you are subscribed to this thread.[Tracking image][https://github.com/notifications/beacon/AKGBAYHSXL4UKDJGCBAUHYTW4RZUZA5CNFSM6AAAAAAQ4LAPQ6WGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTSX3KRKA.gif]
|
I can also confirm that this is the case. My experiment on the gfw-report fork of Shadowsocks vs regular AEAD Shadowsocks has failed miserably because neither got blocked even with very large & consistent downloading traffic. My testing methodology:
I will report back if my AEAD Shadowsocks connection gets blocked. |
This comment was marked as off-topic.
This comment was marked as off-topic.
Thank you @UjuiUjuMandan, I know you mean well, but I would like to respect people's decision to delete their comments. I wrote when this happened before:
|
|
I think the national firewall is currently unable to detect utls, because according to my one-week test, in the case of using xtls-vision and tls, using fallback for web page masquerade, using chrome's utls national firewall is completely unable to detect and block, because x- ray's utls is very conspicuous, exactly the same, which is one of the biggest features, so it is recommended to use utls on the client v2rayn |
|
20230501更新，最近一周vless+ws+tls已被gfw识别并阻断。现象是只要使用xray/v2ray连过服务器，443端口立马被全国屏蔽，浏览器也无法访问服务端web页面，隔一段时间越约20-60分钟后恢复，国外访问一切正常。也就是说gfw能够根据流量特征精准识别tls流量是否是浏览器发出。另外用浏览器频繁重试访问web页面能加快恢复 20230501 update, the last week vless+ws+tls has been identified and blocked by gfw. The phenomenon is as long as the use of xray/v2ray connected to the server, port 443 immediately blocked by the national, the browser also can not access the server web page, after a period of time the more about 20-60 minutes to recover, foreign access to all normal. That is, gfw can accurately identify tls traffic according to traffic characteristics whether the browser is issued. In addition, frequent retries with the browser to access the web page can speed up the recovery |
修改指纹能否复现？ Can modified fingerprints be reproduced? |
我使用的已经是没有指纹问题的版本，V2Ray和XRay都能复现。但是我编写Python、Java程序测试WebSocket连接，都不会被触发gfw，仅V2Ray会触发，无论是TLS1.2还是1.3，无论是VMESS还是VLESS都能被精准识别 I am already using a version without fingerprinting problems, and both V2Ray and XRay are able to reproduce it. But I write Python and Java programs to test WebSocket connections, none of them are triggered by gfw, only V2Ray is triggered, whether TLS1.2 or 1.3, whether VMESS or VLESS can be accurately identified |
|
我估计是流量特征，节点的流量普遍较大，而且websocket可能有了v2ray特征，而且websocket没cdn目前最不安全 I estimate the traffic characteristics, the node traffic is generally larger, and websocket may have a v2ray feature, and websocket no cdn is currently the most insecure |
我后面写程序测试了，只要是WebSocket都会被屏蔽，不管里面是不是V2Ray，而且是在WebSocket的Open帧就立马被RESET。gfw看样子已经能够精确识别TLS加密的WebSocket了。另外我还写程序测试了HTTPS流量里面只要出现TLS握手数据，不管握手数据在报文中任何位置，都是立马被RESET。Trojan-Go的HTTPS流量也是秒被RESET，只要Trojan-Plus使用OpenSSL发出的流量不会被RESET。所以：可以确定三点，gfw能检测出WebSocket Over TLS，能检测处TLS in TLS，能检测出Go程序发起的TLS握手(这点我写Go程序得到了验证)...我的IP已被彻底封禁，不能继续测试了 I wrote a program to test it later, as long as the WebSocket is blocked, regardless of whether it is V2Ray or not, and it is immediately RESET in the Open frame of the WebSocket. gfw seems to be able to accurately identify TLS-encrypted WebSockets now. I also wrote a program to test that HTTPS traffic is immediately RESET whenever there is TLS handshake data, regardless of where the handshake data is in the message. trojan-Go HTTPS traffic is also RESET in seconds, as long as the traffic sent by Trojan-Plus using OpenSSL is not RESET. so: we can be sure of three things. gfw detects WebSocket Over TLS, detects TLS in TLS, and detects TLS handshakes initiated by Go programs (which I verified by writing Go programs)... My IP has been completely blocked and I can't continue testing |
|
@SandroDickens Do you need more IPs? I'd like to sponsor more servers both inside and outside of China for testing. Also can you share your testing code? |
新开瓦工ip，trojan+websocket跑了一个星期并没有出现秒断的情况，不过期间只用quantumult x及自己实现的python脚本连接过 A new Bandwagon ip, trojan + websocket ran for a week and did not appear to break in seconds, but during the period only with quantumult x and their own implementation of the python script to connect over |
This new paper, though, is not about blocking TLS-based tunnels, which is the subject of this thread. The paper is rather about blocking fully encrypted tunnels, like Shadowsocks, VMess, obfs4, etc. See Section 4.3:
Of course, the GFW also blocks certain TLS-based connections, but that is likely handled by a separate module or subsystem in the GFW from the one that blocks fully encrypted protocols. It is likely that there are multiple detection+blocking engines operating somewhat independently. |
Running for months this way. Haven't being blocked yet. |
Could you test this: sending 1~N Rest packets before the handshake and set the TTL is not enough to reach the server but will reach the GFW. |
|
We see increasing number of bans last few days-10 days. It is similar to the one back in September/2022. Do you guys see similar activity on your side please? |
|
Today 20.02.2024 we are experiencing IP ban of our servers. Hundreds of them. Is this related to us or is there a large scale ban going on in China right now? Any feedback on this? |
We do not see this. 3% of our IP's are blocked, which is stable throughout the last 12 months. An important NPC meeting starts on March 5th. What you see may be the typical crack-down prior to such meetings. If you are affected by such a crack-down, in our experience, it's the result of having easily detected VPN protocols on your servers. E.g. PPTP, OpenVPN, IKEv2, Wireguard, etc. |
This comment was marked as off-topic.
This comment was marked as off-topic.
|
We are experiencing currently ip:port ban. Previously whole ip would get banned. But now the ban is ip:port fixed. |
and others
Starting from October 3, 2022 (Beijing Time), more than 100 users reported that at least one of their TLS-based censorship circumvention servers had been blocked. The TLS-based circumvention protocols that are reportedly blocked include trojan, Xray, V2Ray TLS+Websocket, VLESS, and gRPC. We have not received any report of the blocking of naiveproxy though.
Below are a summary of this blocking event and our conjuncture.
The blocking is done by blocking the specific port that the circumvention services listen on. When the user change the blocked port to a non-blocked port and keep using the circumvention tools, the entire IP addresses may get blocked. It is worth noting that their domain names are not added to GFW's DNS or SNI blacklists.
While most of the users report their port 443 got blocked, a few users reported that their non-443 port on which circumvention services listen got blocked as well. While most of the blocked servers are in some popular VPSes providers' datacenters (for example, the bandwagonhost), at least one user reported the blocking of a server in residential network in Europe.
In a few cases (not all cases), the blocking seems to be dynamic because the web browser could still access their circumvention ports but not the circumvention tools did not work.
All these observations above strongly indicate that the GFW can indeed accurately identify and block the circumvention services, rather than simply block the port 443, or block the popular VPS providers.
Based on the information collected above, we suspect, without empirical measurement yet, that the blocking is possibly related to the TLS fingerprints of those circumvention tools. Perhaps developers want to look into uTLS. One may also find this paper reading group, this summary, and this post on TLS fingerprint helpful.
We will investigate if the GFW indeed uses the TLS fingerprints sent by these clients to identify circumvention protocols. At the same time, if you have any server being blocked, or if you have any evidence that can corroborate or falsify our hypothesis, we courage you to share your comments publicly or privately. Our private contact information can be found at the footer of GFW Report.
The text was updated successfully, but these errors were encountered: