XTLS
Feature Request: Custom HTTP Header Manipulation in Xray
Feature Request:
I would like to propose a feature enhancement for Xray regarding the manipulation of HTTP headers. This feature would significantly benefit scenarios where specific bugs in DPI (Deep Packet Inspection) systems can be exploited by customizing HTTP headers.
Current Limitation:
Currently, Xray's handling of HTTP headers seems to have certain limitations. For instance, headers like Host are treated with case sensitivity and follow a predefined placement in the request structure (e.g., the Host header always appears as the second header). This behavior limits the ability to effectively manipulate headers for specific use cases.
Suggested Enhancement:
It would be highly beneficial if Xray could provide functionality similar to curl, where headers can be entered as complete strings, allowing full control over their content, case, and order. Additionally, having an option to disable default headers (such as Host and User-Agent) would further enhance this capability.
This feature would enable more advanced testing and exploitation of DPI system vulnerabilities, where specific header manipulation is required.
Thank you for considering this enhancement. I believe it would add significant value to Xray's functionality in HTTP testing scenarios.
Considering the severity of filtering, especially in Iran, if this helps people, please cooperate
This is a very interesting and important topic, and it will greatly help to remove the limitations in internet filtering.
我强烈推荐实施建议的 Xray 关于 HTTP 头部操作的功能增强。该添加将大大受益于参与高级安全测试和利用深度数据包检测 (DPI) 系统漏洞的用户。
目前，Xray 在头部操作（区分大小写，预定义位置）方面的限制使其在特定用例中的有效性受到限制。建议的功能（模仿 curl 的行为）直接解决了这些限制。允许用户定义带有完全内容、大小写和顺序控制的完整字符串作为头，提供了无与伦比的灵活性。 进一步赋予用户禁用 Host 和 User-Agent 等默认头的选项，甚至可以进一步扩展此功能。
合并此功能将释放 Xray 深入测试和利用的潜力，使用户能够更深入地研究涉及复杂头部操作的漏洞。此增强功能与 Xray 促进全面 HTTP 测试的核心目的完美契合，最终为其功能增加巨大价值。
Considering the severity of filtering, especially in Iran, if this helps people, please cooperate @RPRX
Since every country has different gfw its important to have every possible feature embedded in xray to help all the world fight for freedom uoosef is definitely an expert when it comes to iranian filternet
The Host header has been specially processed to comply with some protocol specifications Because HTTP is rarely used on the public network, the effect of manipulating the header is minimal
The Host header has been specially processed to comply with some protocol specifications Because HTTP is rarely used on the public network, the effect of manipulating the header is minimal
Since every country has different gfw its important to have every possible feature embedded in xray to help all the world fight for freedom uoosef is definitely an expert when it comes to iranian filternet
https://xtls.github.io/config/transports/tcp.html
Based on the outlined capabilities of Xray, specifically its feature set related to TCP transmission mode and HTTP masquerading configuration, it appears that the requested feature enhancement is already encompassed within Xray's existing functionality. Xray provides detailed options for masquerading TCP packets with customizable HTTP headers for both requests and responses. This includes the ability to set the version, method, path, and headers for HTTP requests, as well as the version, status, reason, and headers for HTTP responses.
The HTTP masquerading feature specifically allows for a high degree of control over the headers, including customizing multiple host values and user agents, setting content type, and even manipulating encoding and connection types. Moreover, Xray supports the random selection of path and header values from specified arrays, offering variability in the requests and responses crafted for masquerading purposes.
Therefore, the current Xray functionality already addresses the core aspects of the proposed feature enhancement:
-
Full Control Over Header Content and Ordering: The detailed configuration objects for HTTP requests (
HTTPRequestObject) and responses (HTTPResponseObject) provide the capability to fully control header content, including specifying multiple possible values and selecting among them randomly. This allows for intricate manipulation of HTTP headers to exploit specific vulnerabilities in DPI systems. -
Disabling Default Headers: While the detailed documentation excerpt does not directly mention the ability to explicitly disable default headers such as
Host and User-Agent, the extensive customization provided suggests that users have the option to omit or alter default headers as needed for their specific use cases.
Given these capabilities, the enhancement you've proposed, aimed at increasing flexibility in HTTP header manipulation to exploit DPI bugs through customizations, seems to be largely covered by Xray's existing features. However, if the feature request is aimed at even more granular control or specific functionalities not covered here (for example, explicit disabling of certain default headers without having to provide an alternative), it might be worth clarifying those needs to assess whether further enhancements to Xray could be beneficial.
direct tweaking of HTTP headers might not significantly influence or improve operational efficacy or security in most typical public network scenarios. This could be attributed to the fact that HTTP (without TLS/SSL, i.e., HTTPS) traffic is less commonly used for sensitive or secure communications over the Internet due to its plain-text nature. As such, efforts to manipulate HTTP headers as a means of evading detection or enhancing privacy might have diminishing returns, especially as more sophisticated detection mechanisms (that do not solely rely on header analysis) and increased use of encrypted protocols (HTTPS) become more prevalent.
@us254 @Fangliding @RPRX
The aim of this proposal is to leverage certain features on public CDNs such as Cloudflare, rather than on private servers. For CDNs like Cloudflare, it is not a concern if a DPI (Deep Packet Inspection) system identifies the use of their IP addresses for circumventing filters because, ultimately, they cannot block these IPs without incurring significant costs. However, this method would not be effective for private servers and dedicated IP addresses, as the goal is not to use it directly on a VPS.
Given that Iran's DPI system cannot afford to cut off IPs of public services like Cloudflare or Gcore without significant expenses, the system can only block tunnel connections by detecting the host or SNI. If we can create a scenario where the DPI system cannot recognize HTTP request hosts like 'ws' but the public CDN can, we effectively bypass the DPI system.
There are not a few such bugs in the DPI system, especially regarding HTTP headers, and I have found many such bugs in Iran's DPI system that I believe are not justifiable to discuss publicly.
What we need is the ability to:
- Disable all default headers that Xray adds to HTTP requests.
- Enter headers in a single piece instead of the "Key": "Value" format, such as "Host: example.com", and be able to include special characters like \n without them being stripped.
- Preserve the order of headers set in the config during the sending of requests.
Regarding the security of using HTTP without TLS for tunneling, I believe that when using fully encrypted protocols like vmess, there are no concerns.
@uoosef
DPI system cannot recognize HTTP request hosts like 'ws' but the public CDN can.
I don't understand how this could be possible. Since it's plain text (with very special characteristics), if CDN can read it, so can the firewall, which will obviously be updated to detect new protocols. And I also doubt that if such traffic with non-standard HTTP headers will be accepted by CDN.
@merrkry I won't share specifics about the bugs I've found publicly, as doing so could lead to them being reported and subsequently fixed. Rest assured, I'm not alone in this discovery; numerous individuals have found comparable vulnerabilities in Iran's DPI system. I possess working proofs of concept for some of these issues and am willing to share them with trusted Xray developers, especially those who have access to Iranian carriers like IR-MCI, to allow for direct testing.
I would also like to remind you about the TLS fragmentation technique which, at first glance, might have seemed impossible. Yet experience has shown otherwise, and the feature was eventually implemented. Furthermore, the techniques I'm referring to have been exploited by previous tools such as GoodbyeDPI, GreenTunnel, and many others to deceive the DPI systems effectively.
Outline Client has a feature that allows you to inject arbitrary prefixes (such as HtTPgEt) into the beginning of each connection which has been effective to some extend.
https://www.reddit.com/r/outlinevpn/wiki/index/prefixing/
Some hints are mentioned here: https://github.com/ValdikSS/GoodbyeDPI?tab=readme-ov-file#active-dpi Overall, it would be great to have more granular control on HTTP header manipulation.
Install the go-resty library
go get -u github.com/go-resty/resty/v2
Create a new Go file (e.g., main.go)
package main
import (
"fmt"
"github.com/go-resty/resty/v2"
)
// Response struct to unmarshal JSON response (customize based on your API response structure)
type Response struct {
// Add fields based on the expected response JSON structure
}
func main() {
// Target URL
url := "https://www.example.com"
// Step 1: Create a Resty client
client := resty.New()
// Step 2: Disable default headers like Host and User-Agent
client.SetDisableWarn(true)
client.SetHeader("Host", "")
// Step 3: Create custom headers as complete strings
headers := []string{
"Authorization: Bearer YOUR_ACCESS_TOKEN",
"X-Custom-Header: CustomValue",
"Another-Header: AnotherValue",
}
// Step 4: Add custom headers to the request
for _, header := range headers {
client.SetRawHeader(header)
}
// Step 5: Make a GET request
resp, err := client.R().
SetResult(&Response{}). // Unmarshal response JSON into a struct (customize based on your API response structure)
Get(url)
if err != nil {
fmt.Println("Error making request:", err)
return
}
// Step 6: Access response details
fmt.Println("Status Code:", resp.StatusCode())
fmt.Println("Response Body:", resp.String())
// Step 7: Access and display specific response headers
fmt.Println("Content-Type Header:", resp.Header().Get("Content-Type"))
fmt.Println("Server Header:", resp.Header().Get("Server"))
}
