Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add symmetric mode #52

Merged
merged 1 commit into from
Oct 12, 2023
Merged

Conversation

taitov
Copy link
Contributor

@taitov taitov commented Oct 11, 2023

In the current architecture, each RX queue binds to exactly one worker (one CPU core), and the number of TX queues is equal to the number of all workers (including slowworker). But in some cases, the same number of RX and TX queues is required. For example, virtual device or hardware without RSS.

Now you can specify the option symmetric_mode: true on the port, which will create the same number of RX queues as on TX.

1 core per port:

symmetric_mode: false (default)

port0_rx0 ------> worker0 ---+--> port0_tx0
                  (core0)    \--> port1_tx0
                       
port1_rx0 ------> worker1 ---+--> port0_tx1
                  (core1)    \--> port1_tx1
                       
kernel ------> slowworker ---+--> port0_tx2
               (core2)       \--> port1_tx2
                          
total: RX 1, TX 3 per port
symmetric_mode: true

port0_rx0 ---+--> worker0 ---+--> port0_tx0
port0_rx1 ---|    (core0)    \--> port1_tx0
port0_rx2 ---/
                       
port1_rx0 ---+--> worker1 ---+--> port0_tx1
port1_rx1 ---|    (core1)    \--> port1_tx1
port1_rx2 ---/
        
kernel ------> slowworker ---+--> port0_tx2
               (core2)       \--> port1_tx2

total: RX 3, TX 3 per port

2 cores per port:

symmetric_mode: false (default)

port0_rx0 ------> worker0 ---+--> port0_tx0
                  (core0)    \--> port1_tx0

port0_rx1 ------> worker1 ---+--> port0_tx1
                  (core1)    \--> port1_tx1
                       
port1_rx0 ------> worker2 ---+--> port0_tx2
                  (core2)    \--> port1_tx2

port1_rx1 ------> worker3 ---+--> port0_tx3
                  (core3)    \--> port1_tx3
                       
kernel ------> slowworker ---+--> port0_tx4
               (core4)       \--> port1_tx4
                          
total: RX 2, TX 5 per port
symmetric_mode: true

port0_rx0 ---+--> worker0 ---+--> port0_tx0
port0_rx2 ---|    (core0)    \--> port1_tx0
port0_rx4 ---/

port0_rx1 ---+--> worker1 ---+--> port0_tx1
port0_rx3 ---/    (core1)    \--> port1_tx1
                       
port1_rx0 ---+--> worker2 ---+--> port0_tx2
port1_rx2 ---|    (core2)    \--> port1_tx2
port1_rx4 ---/

port1_rx1 ---+--> worker3 ---+--> port0_tx3
port1_rx3 ---/    (core3)    \--> port1_tx3
                       
kernel ------> slowworker ---+--> port0_tx4
               (core4)       \--> port1_tx4

total: RX 5, TX 5 per port

@taitov taitov force-pushed the symmetric branch 2 times, most recently from 3a91105 to 92146b2 Compare October 11, 2023 07:16
In the current architecture, each RX queue binds to exactly one worker (one CPU core), and the number of TX queues is equal to the number of all workers (including slowworker). But in some cases, the same number of RX and TX queues is required. For example, virtual device or hardware without RSS.

Now you can specify the option symmetric_mode: true on the port, which will create the same number of RX queues as on TX.

1 core per port:

symmetric_mode: false (default)

port0_rx0 ------> worker0 ---+--> port0_tx0
                  (core0)    \--> port1_tx0

port1_rx0 ------> worker1 ---+--> port0_tx1
                  (core1)    \--> port1_tx1

kernel ------> slowworker ---+--> port0_tx2
               (core2)       \--> port1_tx2

total: RX 1, TX 3 per port

symmetric_mode: true

port0_rx0 ---+--> worker0 ---+--> port0_tx0
port0_rx1 ---|    (core0)    \--> port1_tx0
port0_rx2 ---/

port1_rx0 ---+--> worker1 ---+--> port0_tx1
port1_rx1 ---|    (core1)    \--> port1_tx1
port1_rx2 ---/

kernel ------> slowworker ---+--> port0_tx2
               (core2)       \--> port1_tx2

total: RX 3, TX 3 per port


2 cores per port:

symmetric_mode: false (default)

port0_rx0 ------> worker0 ---+--> port0_tx0
                  (core0)    \--> port1_tx0

port0_rx1 ------> worker1 ---+--> port0_tx1
                  (core1)    \--> port1_tx1

port1_rx0 ------> worker2 ---+--> port0_tx2
                  (core2)    \--> port1_tx2

port1_rx1 ------> worker3 ---+--> port0_tx3
                  (core3)    \--> port1_tx3

kernel ------> slowworker ---+--> port0_tx4
               (core4)       \--> port1_tx4

total: RX 2, TX 5 per port

symmetric_mode: true

port0_rx0 ---+--> worker0 ---+--> port0_tx0
port0_rx2 ---|    (core0)    \--> port1_tx0
port0_rx4 ---/

port0_rx1 ---+--> worker1 ---+--> port0_tx1
port0_rx3 ---/    (core1)    \--> port1_tx1

port1_rx0 ---+--> worker2 ---+--> port0_tx2
port1_rx2 ---|    (core2)    \--> port1_tx2
port1_rx4 ---/

port1_rx1 ---+--> worker3 ---+--> port0_tx3
port1_rx3 ---/    (core3)    \--> port1_tx3

kernel ------> slowworker ---+--> port0_tx4
               (core4)       \--> port1_tx4

total: RX 5, TX 5 per port
@GeorgyKirichenko GeorgyKirichenko merged commit be5d25f into yanet-platform:main Oct 12, 2023
@taitov taitov deleted the symmetric branch October 19, 2023 07:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants