蜷缩的蜗牛
蜷缩的蜗牛

Istio故障排除常用指令

kbsonlong 收录于 云原生  Istio
 约 4000 字 预计阅读 8 分钟 - 次阅读  

局部开启 Access 日志 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

export NAMESPACE=default
export WORKLOAD=details
cat << EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: enable-accesslog
  namespace: ${NAMESPACE}
spec:
  configPatches:
  - applyTo: NETWORK_FILTER
    match:
      context: ANY
      listener:
        filterChain:
          filter:
            name: envoy.filters.network.http_connection_manager
    patch:
      operation: MERGE
      value:
        typed_config:
          '@type': type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          access_log:
          - name: envoy.access_loggers.file
            typed_config:
              '@type': type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
              path: /dev/stdout
  workloadSelector:
    labels:
      app: ${WORKLOAD}
EOF

修改 Envoy 日志级别

修改所有 logger 

1
2

export POD_NAME=xxx
kubectl exec -ti -n ${NAMESPACE} ${POD_NAME} -c istio-proxy -- curl -X POST 127.0.0.1:15000/logging\?level=info
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

active loggers:
  admin: info
  alternate_protocols_cache: info
  aws: info
  assert: info
  backtrace: info
  cache_filter: info
  client: info
  config: info
  connection: info
  conn_handler: info
  decompression: info
  dns: info
  dubbo: info
  envoy_bug: info
  ext_authz: info
  rocketmq: info
  file: info
  filter: info
  forward_proxy: info
  grpc: info
  hc: info
  health_checker: info
  http: info
  http2: info
  hystrix: info
  init: info
  io: info
  jwt: info
  kafka: info
  key_value_store: info
  lua: info
  main: info
  matcher: info
  misc: info
  mongo: info
  quic: info
  quic_stream: info
  pool: info
  rbac: info
  redis: info
  router: info
  runtime: info
  stats: info
  secret: info
  tap: info
  testing: info
  thrift: info
  tracing: info
  upstream: info
  udp: info
  wasm: info

修改其中一个 logger 级别 

1

kubectl exec -ti -n ${NAMESPACE} ${POD_NAME} -c istio-proxy -- curl -X POST 127.0.0.1:15000/logging\?http=trace
1
2
3
4
5
6
7
8
9

active loggers:
...
...
  health_checker: warning
  http: trace
  http2: warning
  hystrix: warning
...
...
1
2
3
4
5
6

pod=`kubectl get pod -n istio-system -l app=istiod -o name`
kubectl exec $pod -n istio-system -- curl http://127.0.0.1:15014/debug/$@
kubectl exec $pod -n istio-system -- curl http://127.0.0.1:15014/debug/endpointz
kubectl exec $pod -n istio-system -- curl http://127.0.0.1:15014/debug/adsz
kubectl exec $pod -n istio-system -- curl http://127.0.0.1:15014/debug/registryz
kubectl exec $pod -n istio-system -- curl http://127.0.0.1:15014/debug/configz

常用脚本

由于线上主机初始化没有安装 socat, 在使用istioctl是无法转发端口,所以直接导出 Envoy 的配置，然后在使用 istioctl操作。

导出config 

1
2
3
4

namespace=zsl-test
pod_name=mydemo-my-demo-sgcanshu-4vb7b

kubectl exec -ti -n ${namespace} ${pod_name} -c istio-proxy -- curl http://127.0.0.1:15000/config_dump > config_dump.json

导出cluster,(endpoint) 

1

kubectl exec -ti -n ${namespace} ${pod_name} -c istio-proxy -- curl "http://127.0.0.1:15000/clusters?format=json" > envoy-clusters.json

istioctl 分析文件 

1
2
3
4

istioctl proxy-config listener -f config_dump.json --port 5000
istioctl proxy-config endpoints -f envoy-cluster.json --cluster "outbound|5000||helloworld.sample.svc.cluster.local"
# 过滤存在多版本的Endpoint
istioctl proxy-config endpoints -f envoy-clusters.json | awk -F'[ ]+' '$NF ~ /outbound\|[0-9]+\|[^|]+\|/ {print $0}'

查看sidecar证书是否正常 

1
2
3
4

# ./istioctl proxy-config secret -f config_dump.json
RESOURCE NAME     TYPE           STATUS     VALID CERT     SERIAL NUMBER                               NOT AFTER                NOT BEFORE
default           Cert Chain     ACTIVE     true           219345628773408727192682202221320774018     2023-07-18T01:56:01Z     2023-07-17T01:54:01Z
ROOTCA            CA             ACTIVE     true           17390732964404848583                        2031-12-13T09:40:24Z     2021-12-15T09:40:24Z

查看sidecar证书详细 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77

./istioctl proxy-config secret -f config_dump.json -o json| jq '[.dynamicActiveSecrets[] | select(.name == "default")][0].secret.tlsCertificate.certificateChain.inlineBytes' -r | base64 -d | openssl x509 -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            a5:04:6e:74:53:d1:3e:4c:c9:03:67:67:2e:2e:a1:82
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: O=Istio, CN=Intermediate CA, L=dev
        Validity
            Not Before: Jul 17 01:54:01 2023 GMT
            Not After : Jul 18 01:56:01 2023 GMT
        Subject:
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:ce:0d:56:32:6d:62:c0:27:df:eb:fa:21:03:1d:
                    da:86:44:10:e5:89:5d:b1:2d:05:d7:5b:0d:4f:68:
                    6e:27:39:46:2f:1c:6a:fd:ee:b3:29:e8:d8:5e:db:
                    8c:d3:23:64:71:64:9a:11:d5:65:81:08:d0:55:24:
                    2e:88:1d:08:d3:3c:20:57:a9:c4:9e:fe:8d:4e:8e:
                    81:03:e0:e8:f2:0d:05:13:a5:1b:9b:54:18:73:ea:
                    13:b2:9b:f0:63:34:23:77:eb:db:bc:fb:a8:56:2d:
                    30:be:0c:55:ce:9b:38:ca:ba:05:03:9e:6a:88:26:
                    4f:6c:49:18:00:d0:c7:41:a8:6d:73:96:76:ce:a5:
                    ee:35:98:8a:b7:d0:c1:37:11:1e:3f:ff:42:e2:29:
                    23:9c:3e:0c:ad:9a:70:56:b6:4f:dd:24:a4:17:19:
                    33:c1:7b:cb:4d:2a:10:2a:20:6c:1f:1b:78:55:5a:
                    a1:88:c0:d8:00:65:be:1c:df:5d:30:02:77:0c:c3:
                    df:8e:59:78:2e:a1:31:29:2b:2c:ce:7f:80:74:18:
                    4a:e3:db:34:b3:db:7e:1d:02:15:6d:b1:46:6f:e2:
                    4f:60:c4:d9:1c:e7:29:86:6e:3c:b7:7a:12:e7:38:
                    71:6e:36:ae:46:68:c7:a8:ea:7e:85:1f:6c:3b:0e:
                    ab:81
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage:
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Authority Key Identifier:
                keyid:C0:C7:D4:2D:86:25:EF:D9:AF:0A:76:7B:CB:99:52:FB:67:C7:E0:6E

            X509v3 Subject Alternative Name: critical
                URI:spiffe://cluster.local/ns/zsl-test/sa/default
    Signature Algorithm: sha256WithRSAEncryption
         10:9d:85:38:7f:7f:7b:b2:1a:e1:69:de:68:f9:87:f2:64:c8:
         0b:c6:ab:9c:9b:bf:2a:d3:c6:f1:b8:e1:fd:25:15:cf:05:5d:
         17:73:79:44:fe:dd:b4:28:cf:70:6d:f7:58:91:85:27:33:02:
         7b:95:1a:90:19:0c:1c:7b:03:6a:fa:da:a6:9b:59:a5:23:40:
         4b:03:34:aa:20:65:18:de:8c:14:19:f4:7c:cb:04:83:a0:af:
         c5:43:bf:93:94:ae:51:b7:92:a3:17:b3:10:4f:a0:f4:44:d4:
         b3:24:aa:6b:1e:fd:cb:7a:50:9c:7d:1b:ac:be:dc:b3:2f:1e:
         2f:76:ea:a5:81:a2:a8:ee:0b:d2:a6:cb:15:e4:23:08:66:e3:
         3f:66:c2:ca:9a:71:d2:d9:5e:20:0d:bc:21:21:94:e3:68:1d:
         fb:c8:de:d2:7c:78:68:7d:d1:5f:44:26:98:cc:3c:41:9b:dd:
         cd:5f:73:cb:8a:56:a5:72:e5:21:27:49:a2:8b:3b:1c:06:c3:
         f8:7b:98:10:7c:59:b2:6c:05:4e:ec:eb:d8:9f:33:19:4b:3e:
         98:08:17:fa:bf:1a:3a:b7:76:8e:c2:7b:84:be:d2:f8:b8:55:
         b1:f9:72:7b:e5:91:50:1a:09:bf:1c:b3:05:72:53:7b:f4:e2:
         bd:66:d3:7e:5c:70:50:8b:b1:91:04:30:d5:52:74:fd:2a:18:
         56:7d:a2:83:a9:25:d7:d4:0d:f4:45:5f:1f:a2:bd:52:7e:bf:
         1b:99:92:63:20:cb:6f:18:24:83:4b:fa:1b:ab:49:44:11:58:
         c2:ca:9f:4e:c4:8e:ea:18:09:a7:84:6b:75:ef:ea:fd:54:d9:
         1e:36:b2:89:70:35:fb:f1:b3:c5:5e:e5:75:7b:a4:8b:1d:f6:
         3b:13:92:90:de:09:3d:15:dd:e8:3c:e9:e6:50:28:17:88:85:
         12:c4:80:c9:fe:ca:10:ae:eb:37:ed:5c:13:a4:f5:0d:6e:41:
         4f:84:b3:94:1e:ea:a0:52:47:1d:6c:8e:fe:f5:1e:91:0e:7a:
         fe:31:17:97:b5:04:1c:d4:27:af:d1:df:3f:e4:38:2d:78:11:
         aa:d0:54:50:e8:4d:bf:10:55:7b:3d:29:18:e4:a8:f1:f7:e5:
         c1:b1:41:de:6c:8b:d1:0e:ca:2f:69:5d:b8:08:da:fa:30:f9:
         1a:5f:c2:93:56:52:1d:39:41:f7:0c:c9:c7:d7:bc:33:99:cc:
         91:52:95:0b:35:0b:a9:10:29:66:0a:93:4e:50:ee:83:25:4c:
         92:ff:6f:94:3d:f2:39:9c:3a:5b:0a:38:c1:c5:bd:50:b8:4d:
         27:ea:f9:8f:89:e4:4c:8a

常用yaml

sidecar 注入相关

为指定 workload 取消 sidecar 自动注入

1

kubectl label ns zsl-test istio-injection=enabled
1
2
3
4

  template:
    metadata:
      annotations:
        sidecar.istio.io/inject: "false"

proxy 相关

自定义资源 

1
2
3
4
5
6
7

  template:
    metadata:
      annotations:
        "sidecar.istio.io/proxyCPU": "10m"
        "sidecar.istio.io/proxyCPULimit": "2"
        "sidecar.istio.io/proxyMemory": "32Mi"
        "sidecar.istio.io/proxyMemoryLimit": "1Gi"

自定义日志级别 

1
2
3
4
5

  template:
    metadata:
      annotations:
        "sidecar.istio.io/logLevel": debug # 可选: trace, debug, info, warning, error, critical, off
        "sidecar.istio.io/componentLogLevel": "ext_authz:trace,filter:debug"

不劫持部分流量 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

  template:
    metadata:
      annotations:
        traffic.sidecar.istio.io/excludeOutboundIPRanges: "10.10.31.1/32,10.10.31.2/32" # 不劫持的出站目标地址
        traffic.sidecar.istio.io/excludeInterfaces: ""  # 过滤网卡接口
        traffic.sidecar.istio.io/excludeInboundPorts: ""  # 不劫持的入站端口
        traffic.sidecar.istio.io/excludeOutboundPorts: "" # 不劫持的出站端口
        traffic.sidecar.istio.io/includeInboundPorts: "" # 只劫持指定入站端口
        traffic.sidecar.istio.io/includeOutboundPorts: "" # 不管任意IP,指定出站端口的流量都将被重定向到Envoy
        traffic.sidecar.istio.io/includeOutboundIPRanges: "" # 只劫持指定出站目标地址

istio 流量劫持是通过 iptable 来实现。以上 annotations 都是通过 istio-init 初始化容器操作,当服务开启sidecar注入之后，发现流量异常为避免服务重启，可以通过修改iptable规则来紧急跳过sidecar流量劫持。 iptable 使用 iptables-legacy 替换

不劫持指定出站端口(443)流量

traffic.sidecar.istio.io/excludeInboundPorts: 443

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

# Generated by iptables-save v1.8.7 on Wed Feb 26 18:35:36 2025
*raw
:PREROUTING ACCEPT [682:2361353]
:OUTPUT ACCEPT [559:720418]
-A PREROUTING -d 169.254.20.10/32 -p udp -m udp --sport 53 -j CT --zone 1
-A OUTPUT -p udp -m udp --dport 53 -m owner --uid-owner 1337 -j CT --zone 1
-A OUTPUT -p udp -m udp --sport 15053 -m owner --uid-owner 1337 -j CT --zone 2
-A OUTPUT -p udp -m udp --dport 53 -m owner --gid-owner 1337 -j CT --zone 1
-A OUTPUT -p udp -m udp --sport 15053 -m owner --gid-owner 1337 -j CT --zone 2
-A OUTPUT -d 169.254.20.10/32 -p udp -m udp --dport 53 -j CT --zone 2
COMMIT
# Completed on Wed Feb 26 18:35:36 2025
# Generated by iptables-save v1.8.7 on Wed Feb 26 18:35:36 2025
*nat
:PREROUTING ACCEPT [63:7951]
:INPUT ACCEPT [63:7951]
:OUTPUT ACCEPT [52:4025]
:POSTROUTING ACCEPT [56:4359]
:ISTIO_INBOUND - [0:0]
:ISTIO_IN_REDIRECT - [0:0]
:ISTIO_OUTPUT - [0:0]
:ISTIO_REDIRECT - [0:0]
-A PREROUTING -p tcp -j ISTIO_INBOUND
-A OUTPUT -p tcp -j ISTIO_OUTPUT
-A OUTPUT -p udp -m udp --dport 53 -m owner --uid-owner 1337 -j RETURN
-A OUTPUT -p udp -m udp --dport 53 -m owner --gid-owner 1337 -j RETURN
-A OUTPUT -d 169.254.20.10/32 -p udp -m udp --dport 53 -j REDIRECT --to-ports 15053
-A ISTIO_INBOUND -p tcp -m tcp --dport 15008 -j RETURN
-A ISTIO_INBOUND -p tcp -m tcp --dport 15090 -j RETURN
-A ISTIO_INBOUND -p tcp -m tcp --dport 15021 -j RETURN
-A ISTIO_INBOUND -p tcp -m tcp --dport 15020 -j RETURN
-A ISTIO_INBOUND -p tcp -j ISTIO_IN_REDIRECT
-A ISTIO_IN_REDIRECT -p tcp -j REDIRECT --to-ports 15006
-A ISTIO_OUTPUT -p tcp -m tcp --dport 443 -j RETURN
-A ISTIO_OUTPUT -s 127.0.0.6/32 -o lo -j RETURN
-A ISTIO_OUTPUT ! -d 127.0.0.1/32 -o lo -p tcp -m tcp ! --dport 53 -m owner --uid-owner 1337 -j ISTIO_IN_REDIRECT
-A ISTIO_OUTPUT -o lo -p tcp -m tcp ! --dport 53 -m owner ! --uid-owner 1337 -j RETURN
-A ISTIO_OUTPUT -m owner --uid-owner 1337 -j RETURN
-A ISTIO_OUTPUT ! -d 127.0.0.1/32 -o lo -m owner --gid-owner 1337 -j ISTIO_IN_REDIRECT
-A ISTIO_OUTPUT -o lo -p tcp -m tcp ! --dport 53 -m owner ! --gid-owner 1337 -j RETURN
-A ISTIO_OUTPUT -m owner --gid-owner 1337 -j RETURN
-A ISTIO_OUTPUT -d 169.254.20.10/32 -p tcp -m tcp --dport 53 -j REDIRECT --to-ports 15053
-A ISTIO_OUTPUT -d 127.0.0.1/32 -j RETURN
-A ISTIO_OUTPUT -j ISTIO_REDIRECT
-A ISTIO_REDIRECT -p tcp -j REDIRECT --to-ports 15001
COMMIT
# Completed on Wed Feb 26 18:35:36 2025
劫持指定出站(80)端口

traffic.sidecar.istio.io/includeOutboundPorts: 80

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

# Generated by iptables-save v1.8.7 on Wed Feb 26 18:44:24 2025
*raw
:PREROUTING ACCEPT [1145:4103074]
:OUTPUT ACCEPT [879:1066596]
-A PREROUTING -d 169.254.20.10/32 -p udp -m udp --sport 53 -j CT --zone 1
-A OUTPUT -p udp -m udp --dport 53 -m owner --uid-owner 1337 -j CT --zone 1
-A OUTPUT -p udp -m udp --sport 15053 -m owner --uid-owner 1337 -j CT --zone 2
-A OUTPUT -p udp -m udp --dport 53 -m owner --gid-owner 1337 -j CT --zone 1
-A OUTPUT -p udp -m udp --sport 15053 -m owner --gid-owner 1337 -j CT --zone 2
-A OUTPUT -d 169.254.20.10/32 -p udp -m udp --dport 53 -j CT --zone 2
COMMIT
# Completed on Wed Feb 26 18:44:24 2025
# Generated by iptables-save v1.8.7 on Wed Feb 26 18:44:24 2025
*nat
:PREROUTING ACCEPT [106:12597]
:INPUT ACCEPT [106:12597]
:OUTPUT ACCEPT [68:5483]
:POSTROUTING ACCEPT [73:5918]
:ISTIO_INBOUND - [0:0]
:ISTIO_IN_REDIRECT - [0:0]
:ISTIO_OUTPUT - [0:0]
:ISTIO_REDIRECT - [0:0]
-A PREROUTING -p tcp -j ISTIO_INBOUND
-A OUTPUT -p tcp -j ISTIO_OUTPUT
-A OUTPUT -p udp -m udp --dport 53 -m owner --uid-owner 1337 -j RETURN
-A OUTPUT -p udp -m udp --dport 53 -m owner --gid-owner 1337 -j RETURN
-A OUTPUT -d 169.254.20.10/32 -p udp -m udp --dport 53 -j REDIRECT --to-ports 15053
-A ISTIO_INBOUND -p tcp -m tcp --dport 15008 -j RETURN
-A ISTIO_INBOUND -p tcp -m tcp --dport 15090 -j RETURN
-A ISTIO_INBOUND -p tcp -m tcp --dport 15021 -j RETURN
-A ISTIO_INBOUND -p tcp -m tcp --dport 15020 -j RETURN
-A ISTIO_INBOUND -p tcp -j ISTIO_IN_REDIRECT
-A ISTIO_IN_REDIRECT -p tcp -j REDIRECT --to-ports 15006
-A ISTIO_OUTPUT -p tcp -m tcp --dport 443 -j RETURN
-A ISTIO_OUTPUT -s 127.0.0.6/32 -o lo -j RETURN
-A ISTIO_OUTPUT ! -d 127.0.0.1/32 -o lo -p tcp -m tcp ! --dport 53 -m owner --uid-owner 1337 -j ISTIO_IN_REDIRECT
-A ISTIO_OUTPUT -o lo -p tcp -m tcp ! --dport 53 -m owner ! --uid-owner 1337 -j RETURN
-A ISTIO_OUTPUT -m owner --uid-owner 1337 -j RETURN
-A ISTIO_OUTPUT ! -d 127.0.0.1/32 -o lo -m owner --gid-owner 1337 -j ISTIO_IN_REDIRECT
-A ISTIO_OUTPUT -o lo -p tcp -m tcp ! --dport 53 -m owner ! --gid-owner 1337 -j RETURN
-A ISTIO_OUTPUT -m owner --gid-owner 1337 -j RETURN
-A ISTIO_OUTPUT -d 169.254.20.10/32 -p tcp -m tcp --dport 53 -j REDIRECT --to-ports 15053
-A ISTIO_OUTPUT -d 127.0.0.1/32 -j RETURN
-A ISTIO_OUTPUT -p tcp -m tcp --dport 80 -j ISTIO_REDIRECT
-A ISTIO_OUTPUT -j ISTIO_REDIRECT
-A ISTIO_REDIRECT -p tcp -j REDIRECT --to-ports 15001
COMMIT
# Completed on Wed Feb 26 18:44:24 2025
  • 默认所有流量都被重定向到 Envoy Port,默认15001
1
2
3
4
5
6
7
8
9

// https://github.com/kbsonlong/istio/blob/1.13.4/tools/istio-iptables/pkg/capture/run.go#L370
// -A ISTIO_REDIRECT -p tcp -j REDIRECT --to-ports 15001


// Create a new chain for redirecting outbound traffic to the common Envoy port.
// In both chains, '-j RETURN' bypasses Envoy and '-j ISTIOREDIRECT'
// redirects to Envoy.
cfg.iptables.AppendRule(iptableslog.UndefinedCommand,
		constants.ISTIOREDIRECT, constants.NAT, "-p", constants.TCP, "-j", constants.REDIRECT, "--to-ports", cfg.cfg.ProxyPort)
劫持指定出站网段 
1

traffic.sidecar.istio.io/includeOutboundIPRanges: 192.168.0.0/24
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

# iptables-legacy-save
# Generated by iptables-save v1.8.7 on Thu Feb 27 17:43:36 2025
*raw
:PREROUTING ACCEPT [1355:3778929]
:OUTPUT ACCEPT [1038:734741]
-A PREROUTING -d 169.254.20.10/32 -p udp -m udp --sport 53 -j CT --zone 1
-A OUTPUT -p udp -m udp --dport 53 -m owner --uid-owner 1337 -j CT --zone 1
-A OUTPUT -p udp -m udp --sport 15053 -m owner --uid-owner 1337 -j CT --zone 2
-A OUTPUT -p udp -m udp --dport 53 -m owner --gid-owner 1337 -j CT --zone 1
-A OUTPUT -p udp -m udp --sport 15053 -m owner --gid-owner 1337 -j CT --zone 2
-A OUTPUT -d 169.254.20.10/32 -p udp -m udp --dport 53 -j CT --zone 2
COMMIT
# Completed on Thu Feb 27 17:43:36 2025
# Generated by iptables-save v1.8.7 on Thu Feb 27 17:43:36 2025
*nat
:PREROUTING ACCEPT [131:13456]
:INPUT ACCEPT [131:13456]
:OUTPUT ACCEPT [67:5370]
:POSTROUTING ACCEPT [67:5382]
:ISTIO_INBOUND - [0:0]
:ISTIO_IN_REDIRECT - [0:0]
:ISTIO_OUTPUT - [0:0]
:ISTIO_REDIRECT - [0:0]
-A PREROUTING -p tcp -j ISTIO_INBOUND
-A OUTPUT -p tcp -j ISTIO_OUTPUT
-A OUTPUT -p udp -m udp --dport 53 -m owner --uid-owner 1337 -j RETURN
-A OUTPUT -p udp -m udp --dport 53 -m owner --gid-owner 1337 -j RETURN
-A OUTPUT -d 169.254.20.10/32 -p udp -m udp --dport 53 -j REDIRECT --to-ports 15053
-A ISTIO_INBOUND -p tcp -m tcp --dport 15008 -j RETURN
-A ISTIO_INBOUND -p tcp -m tcp --dport 15090 -j RETURN
-A ISTIO_INBOUND -p tcp -m tcp --dport 15021 -j RETURN
-A ISTIO_INBOUND -p tcp -m tcp --dport 15020 -j RETURN
-A ISTIO_INBOUND -p tcp -j ISTIO_IN_REDIRECT
-A ISTIO_IN_REDIRECT -p tcp -j REDIRECT --to-ports 15006
-A ISTIO_OUTPUT -p tcp -m tcp --dport 443 -j RETURN
-A ISTIO_OUTPUT -s 127.0.0.6/32 -o lo -j RETURN
-A ISTIO_OUTPUT ! -d 127.0.0.1/32 -o lo -p tcp -m tcp ! --dport 53 -m owner --uid-owner 1337 -j ISTIO_IN_REDIRECT
-A ISTIO_OUTPUT -o lo -p tcp -m tcp ! --dport 53 -m owner ! --uid-owner 1337 -j RETURN
-A ISTIO_OUTPUT -m owner --uid-owner 1337 -j RETURN
-A ISTIO_OUTPUT ! -d 127.0.0.1/32 -o lo -m owner --gid-owner 1337 -j ISTIO_IN_REDIRECT
-A ISTIO_OUTPUT -o lo -p tcp -m tcp ! --dport 53 -m owner ! --gid-owner 1337 -j RETURN
-A ISTIO_OUTPUT -m owner --gid-owner 1337 -j RETURN
-A ISTIO_OUTPUT -d 169.254.20.10/32 -p tcp -m tcp --dport 53 -j REDIRECT --to-ports 15053
-A ISTIO_OUTPUT -d 127.0.0.1/32 -j RETURN
-A ISTIO_OUTPUT -p tcp -m tcp --dport 80 -j ISTIO_REDIRECT
-A ISTIO_OUTPUT -d 192.168.0.0/24 -j ISTIO_REDIRECT
-A ISTIO_OUTPUT -j RETURN
-A ISTIO_REDIRECT -p tcp -j REDIRECT --to-ports 15001
COMMIT
# Completed on Thu Feb 27 17:43:36 2025

includeOutboundIPRanges源码,默认所有流量都被重定向到Envoy,当存在合法CIDR 时,其他流量都不被重定向ISTIO_REDIRECT,都被RETURN回去

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

// https://github.com/kbsonlong/istio/blob/1.13.4/tools/istio-iptables/pkg/capture/run.go#L209
func (cfg *IptablesConfigurator) handleOutboundIncludeRules(
	rangeInclude NetworkRange,
	appendRule func(command iptableslog.Command, chain string, table string, params ...string) *builder.IptablesBuilder,
	insert func(command iptableslog.Command, chain string, table string, position int, params ...string) *builder.IptablesBuilder) {
	// Apply outbound IP inclusions.
	if rangeInclude.IsWildcard {
		// Wildcard specified. Redirect all remaining outbound traffic to Envoy.
    // -A ISTIO_OUTPUT -j ISTIO_REDIRECT
		appendRule(iptableslog.UndefinedCommand, constants.ISTIOOUTPUT, constants.NAT, "-j", constants.ISTIOREDIRECT)
		for _, internalInterface := range split(cfg.cfg.KubeVirtInterfaces) {
			insert(iptableslog.KubevirtCommand,
				constants.PREROUTING, constants.NAT, 1, "-i", internalInterface, "-j", constants.ISTIOREDIRECT)
		}
	} else if len(rangeInclude.IPNets) > 0 {
		// User has specified a non-empty list of cidrs to be redirected to Envoy.
		for _, cidr := range rangeInclude.IPNets {
			for _, internalInterface := range split(cfg.cfg.KubeVirtInterfaces) {
				insert(iptableslog.KubevirtCommand, constants.PREROUTING, constants.NAT, 1, "-i", internalInterface,
					"-d", cidr.String(), "-j", constants.ISTIOREDIRECT)
			}
			appendRule(iptableslog.UndefinedCommand,
				constants.ISTIOOUTPUT, constants.NAT, "-d", cidr.String(), "-j", constants.ISTIOREDIRECT)
		}
		// All other traffic is not redirected.
    // -A ISTIO_OUTPUT -j RETURN
		appendRule(iptableslog.UndefinedCommand, constants.ISTIOOUTPUT, constants.NAT, "-j", constants.RETURN)
	}
}
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59

# 
nsenter -t 126481 -n
iptables-legacy-save
iptables-legacy -t nat -I ISTIO_OUTPUT -p tcp -m tcp --dport 4033 -j RETURN


# 不劫持指定入站端口(80)流量
traffic.sidecar.istio.io/excludeInboundPorts: "80"

iptables-legacy -t nat -I ISTIO_OUTPUT -p tcp -m tcp --dport 80 -j ISTIO_REDIRECT
iptables-legacy -t nat -nL ISTIO_OUTPUT --line-numbers
# 删除第一条规则
iptables-legacy -t nat -D ISTIO_OUTPUT 1


# 不劫持指定出站端口(443)流量
traffic.sidecar.istio.io/excludeOutboundPorts: "443"

iptables-legacy -t nat -I ISTIO_OUTPUT -p tcp -m tcp --dport 443 -j RETURN
iptables-legacy -t nat -nL ISTIO_OUTPUT --line-numbers
# 删除第一条规则
iptables-legacy -t nat -D ISTIO_OUTPUT 1

# 只劫持指定入站端口(8443)流量
traffic.sidecar.istio.io/includeInboundPorts: "8443"

iptables-legacy -t nat -I ISTIO_INBOUND -p tcp -m tcp --dport 8443 -j ISTIO_IN_REDIRECT
iptables-legacy -t nat -nL ISTIO_INBOUND --line-numbers
# 删除第一条规则
iptables-legacy -t nat -D ISTIO_INBOUND 1

# 不劫持的出站目标地址(114.114.114.114/32)流量
traffic.sidecar.istio.io/excludeOutboundIPRanges: 114.114.114.114/32

iptables-legacy -t nat -I ISTIO_OUTPUT -d 114.114.114.114/32 -j RETURN
iptables-legacy -t nat -nL ISTIO_OUTPUT --line-numbers
# 删除第一条规则
iptables-legacy -t nat -D ISTIO_OUTPUT 1

# 不管任意IP,80端口流量都将被重定向到Envoy
traffic.sidecar.istio.io/includeOutboundPorts: "80"

iptables-legacy -t nat -I ISTIO_OUTPUT -p tcp -m tcp --dport 80 -j ISTIO_REDIRECT

# 只劫持指定出站目标地址(8.8.8.8/32)流量
traffic.sidecar.istio.io/includeOutboundIPRanges: 8.8.8.8/32

iptables-legacy -t nat -I ISTIO_OUTPUT -d 8.8.8.8/32 -j ISTIO_REDIRECT
iptables-legacy -t nat -nL ISTIO_OUTPUT --line-numbers
# 删除第一条规则
iptables-legacy -t nat -D ISTIO_OUTPUT 1

# 需要劫持指定出站端口,其他流量不被劫持到Envoy
traffic.sidecar.istio.io/includeOutboundPorts: "80"
traffic.sidecar.istio.io/includeOutboundIPRanges: 192.168.0.1/32

-A ISTIO_OUTPUT -p tcp -m tcp --dport 80 -j ISTIO_REDIRECT
-A ISTIO_OUTPUT -d 192.168.0.1/32 -j ISTIO_REDIRECT
-A ISTIO_OUTPUT -j RETURN

当单独设置 traffic.sidecar.istio.io/includeOutboundPorts 时会跟预想存在区别

因为默认全部流量都被劫持到Envoy, -A ISTIO_OUTPUT -j ISTIO_REDIRECT,

当设置指定IP地址段拦截 traffic.sidecar.istio.io/includeOutboundIPRanges 时,其他流量都不被重定向

-A ISTIO_OUTPUT -j RETURN,所以当需要劫持指定出站端口,并且其他流量不被劫持时两者配合使用可以达到奇效。

全局禁用mTLS 

1
2
3
4
5
6
7
8

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: istio-system
spec:
  mtls:
    mode: DISABLE

常用EnvoyFilter

请求限制 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: http-options
  namespace: istio-system
spec:
  configPatches:
  - applyTo: NETWORK_FILTER
    match:
      context: ANY
      listener:
        filterChain:
          filter:
            name: "envoy.filters.network.http_connection_manager"
    patch:
      operation: MERGE
      value:
        typed_config:
          "@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager"
          max_request_headers_kb: 96 # 96KB, 请求 header 最大限制
  - applyTo: HTTP_FILTER
    match:
      context: GATEWAY
      listener:
        filterChain:
          filter:
            name: "envoy.filters.network.http_connection_manager"
    patch:
      operation: INSERT_BEFORE
      value:
        name: "envoy.filters.http.buffer"
        typed_config:
          '@type': "type.googleapis.com/envoy.extensions.filters.http.buffer.v3.Buffer"
          max_request_bytes: 1048576  # 1MB, 请求最大限制

保留请求头大小写 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: header-casing
  namespace: istio-system
spec:
  configPatches:
    - applyTo: CLUSTER
      match:
        context: SIDECAR_INBOUND
      patch:
        operation: MERGE
        value:
          typed_extension_protocol_options:
            envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
              '@type': type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
              explicit_http_config:
                http_protocol_options:
                  header_key_format:
                    stateful_formatter:
                      name: preserve_case
                      typed_config:
                        '@type': type.googleapis.com/envoy.extensions.http.header_formatters.preserve_case.v3.PreserveCaseFormatterConfig
    - applyTo: NETWORK_FILTER
      match:
        listener:
          filterChain:
            filter:
              name: envoy.filters.network.http_connection_manager
      patch:
        operation: MERGE
        value:
          typed_config:
            '@type': type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
            http_protocol_options:
              header_key_format:
                stateful_formatter:
                  name: preserve_case
                  typed_config:
                    '@type': type.googleapis.com/envoy.extensions.http.header_formatters.preserve_case.v3.PreserveCaseFormatterConfig

删除header

  • 全局删除
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

cat << EOF | kubectl apply -f - 
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: remove-header
  namespace: istio-system
spec:
  configPatches:
  - applyTo: NETWORK_FILTER
    match:
      listener:
        filterChain:
          filter:
            name: "envoy.filters.network.http_connection_manager"
    patch:
      operation: MERGE
      value:
        typed_config:
          "@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager"
          server_header_transformation: PASS_THROUGH
  - applyTo: ROUTE_CONFIGURATION
    patch:
      operation: MERGE
      value:
        request_headers_to_remove:
        - "x-envoy-peer-metadata"
        response_headers_to_remove:
        - "x-envoy-upstream-service-time"
EOF

高频使用链接

istio 相关

Envoy 相关

更新于 2023-03-31 
CC BY-NC 4.0
阅读原始文档
LINE Myspace
FaqIstio
返回 | 主页
0%