vMX running in eve-ng (community)

[juan]

Hi guys,

Do you use vMX to make your labs? I have lot of CPU usage , even turning fpc into "lite" mode.

Do you use some alternative to eve-ng? Perhaps running directly vMX as separete VMs in VMware and connecting them with virtual adapters is a better approach?

Thank you for you advise

juan

[kechik]

Hi,

 

If your pc just spec below 8vcpu and below 16g ram then u cannot do anything.

 

Thanks

[juan]

5 minutes ago, kechik said:

Hi,

 

If your pc just spec below 8vcpu and below 16g ram then u cannot do anything.

 

Thanks

I have a 6 core / 12 threads i7 @4.5GHz and 32GB of RAM.

[juan]

I've managed to run 10 vMX nodes at a time. There are some considerations in order to achieve this. And of course is not perfect, for instance you can get a laggy CLI sometimes. So far I just checked every control plane and PFE is up and running and linked properly, as well as a p2p links connecting all the topology.

1. You can seize vCPU consumption "stopping" Data Plane Development Kit (DPDK) from polling constantly from VCP to VFP. This is accomplished whether running a lab or not, coz is done at Linux VM hosting eve-ng. Here you go the web site which instructs how to make the process. This is intended for JUNIPER devices, not tested with multivendor.

Spoiler

This is the hidden content, please

• Sign In
• or
• Sign Up

2. Limit CPU usage of VMWare: even though VMWare will take over all resources, you can modify settings of eve-ng VM so that it does not feed up you computer resources, thereby avoiding a bottleneck. In my case, I have 6 core and 12 threads. In eve-ng VM settings, I set 3 CPUs and 3 threads per CPU, resulting in 9 threads. This maintains my computer CPU at some 85-90% at maximun.

3. License your vMX devices and enable network-services ip. this will be need for MPLS VPNs and so on... (advances features mostly in PE nodes). I have posted in other thread a WA to install SA license in vMX v19.4

4. Getting rid of bothering syslog messages flooding all time in your console (really Telnet) terminal. This can be done following a WA posted in next website (post no. 😎

Spoiler

This is the hidden content, please

• Sign In
• or
• Sign Up

This is the hidden content, please

• Sign In
• or
• Sign Up

Essentially setting this two commands:

set system syslog user * any emergency
set system syslog user * match "!(.*Scheduler Oinker*.|.*Frame 0*.|.*ms without yielding*.)"

5. Set chassis in lite-mode: in order to expend less vRAM, you can set VCP to 1 vCPU and 1024MB of vRAM and VFP to 3 vCPU and 2048MB of vRAM. Boot with these settings and set the command: "set chassis fpc 0 lite-mode". Once done, reboot. After rebooting check "RIOT LITE-MODE" is on checking show chassis hardware. The essentially decrease the amount of vRAM consumed by vMX. Only drawback is links capacity lowers to 100Mbps. Still enough to make tests with a traffic generator like Ostinato (also supported in eve-ng).

Spoiler

This is the hidden content, please

• Sign In
• or
• Sign Up

I will update this thread if I found more improvements.

Cheers

juan

Edited August 22, 2021 by juan
lite-mode for chassis

[juan]

Added chassis working mode in "lite-mode"

[xttbb]

thanks

[damimad]

THX

[192x168]

On 8/21/2021 at 7:01 PM, juan said:

Do you use vMX to make your labs? I have lot of CPU usage , even turning fpc into "lite" mode.

Do you use some alternative to eve-ng? Perhaps running directly vMX as separete VMs in VMware and connecting them with virtual adapters is a better approach?

vMX 18.2 on pure Linux (no VMs/nested virtualization); GNS3; "lite" mode; laptop with two-core 2.4GHz CPU (HT disabled) and 16GB of RAM; "halt_poll_ns=0".

Overall CPU utilization is 35..40% at idle.

I totally agree with @kechik, you (and me) can't do anything, Juniper software is just really CPU-intensive. An overhead of nested virtualization (using of EVE-NG or GNS3 VM) is way too small to be noticeable. And IBM wrote about "halt_poll_ns=0": "In a environment where CPU resources are highly over-committed, this setting might be counterproductive." 

In my labs I extensively use logical systems and routing instances, so one vMX node is enough to emulate 10..20 routers. I understand that logical systems are not always suitable, that's just my case. vMX 14.1 is quite lightweight, but it lacks all this EVPN bells and whistles, and some configuration commands are a bit different, so it's not really useful now. That's my two cents. And, by the way, thank you for sharing vMX 21.2:)

[nesh]

hh

[Mist]

I always prefer to use VMX/vSRX as separate vm. It gives me a better performance.

12core/24vcpu

64GB RAM

Thanks

[hasanlludi]

thnx

[ciscolessons]

Thank you!

[ciscolessons]

Thank you!

[HH0312]

thanks

[ayodejad]

ok

[angelsgg]

thanks

[Pit101]

On 8/21/2021 at 7:54 PM, juan said:

I've managed to run 10 vMX nodes at a time. There are some considerations in order to achieve this. And of course is not perfect, for instance you can get a laggy CLI sometimes. So far I just checked every control plane and PFE is up and running and linked properly, as well as a p2p links connecting all the topology.

1. You can seize vCPU consumption "stopping" Data Plane Development Kit (DPDK) from polling constantly from VCP to VFP. This is accomplished whether running a lab or not, coz is done at Linux VM hosting eve-ng. Here you go the web site which instructs how to make the process. This is intended for JUNIPER devices, not tested with multivendor.

  Hide contents

 

Hidden Content

• Give reaction or reply to this topic to see the hidden content.

 

2. Limit CPU usage of VMWare: even though VMWare will take over all resources, you can modify settings of eve-ng VM so that it does not feed up you computer resources, thereby avoiding a bottleneck. In my case, I have 6 core and 12 threads. In eve-ng VM settings, I set 3 CPUs and 3 threads per CPU, resulting in 9 threads. This maintains my computer CPU at some 85-90% at maximun.

3. License your vMX devices and enable network-services ip. this will be need for MPLS VPNs and so on... (advances features mostly in PE nodes). I have posted in other thread a WA to install SA license in vMX v19.4

4. Getting rid of bothering syslog messages flooding all time in your console (really Telnet) terminal. This can be done following a WA posted in next website (post no. 😎

  Reveal hidden contents

 

Hidden Content

• Give reaction or reply to this topic to see the hidden content.

 

Hidden Content

• Give reaction or reply to this topic to see the hidden content.

 

 

Essentially setting this two commands:

set system syslog user * any emergency
set system syslog user * match "!(.*Scheduler Oinker*.|.*Frame 0*.|.*ms without yielding*.)"

5. Set chassis in lite-mode: in order to expend less vRAM, you can set VCP to 1 vCPU and 1024MB of vRAM and VFP to 3 vCPU and 2048MB of vRAM. Boot with these settings and set the command: "set chassis fpc 0 lite-mode". Once done, reboot. After rebooting check "RIOT LITE-MODE" is on checking show chassis hardware. The essentially decrease the amount of vRAM consumed by vMX. Only drawback is links capacity lowers to 100Mbps. Still enough to make tests with a traffic generator like Ostinato (also supported in eve-ng).

  Reveal hidden contents

 

Hidden Content

• Give reaction or reply to this topic to see the hidden content.

 

I will update this thread if I found more improvements.

Cheers

juan

thanks