Skip to content

Network Extension feature doesn't persist the vlan information #14077

Description

@kiranchavala

problem

Network Extension feature doesn't persist the vlan information

versions

ACS 4.23, KVM

The steps to reproduce the bug

Follow the steps mentioned here to register network extension

https://github.com/apache/cloudstack-extensions/blob/network-namespace/Network-Namespace/README.md

  1. Create a network offering with vlan enabled

  2. Create a network based on the above network offering

  3. Launch a vm in the network > Vm deployed successfully

  4. Destroy the vm

After some time the network state changes from implemented to allocated since there are no vm's

  1. Check the networks table, the brodcast_uri changes to NULL

  2. The subsequent vm launch will result in a failure

logs

root@Cloudstack-423:/var/log/cloudstack/management#  cat /var/log/cloudstack/management/management-server.log.2026-09-07 | grep -i "logid:63df11b4" | grep -iE "Script output|failed with exit code|Cannot find|Ensuring network
    device|Implementing network extension|applyFWRules|Applying|dhcp|dns"
2026-09-07 11:14:23,762 DEBUG [c.c.c.CapacityManagerImpl] (API-Job-Executor-49:[ctx-564cbc63, job-1216, ctx-099a3943, FirstFitRoutingAllocator]) (logid:63df11b4) Hosts's actual total CPU: 12748 and CPU after applying overprovisioning: 12748
2026-09-07 11:14:24,714 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-63:[ctx-806840fb, job-1216/job-1217, ctx-f951d8a4]) (logid:63df11b4) Hosts's actual total CPU: 12748 and CPU after applying overprovisioning: 12748
2026-09-07 11:14:24,716 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-63:[ctx-806840fb, job-1216/job-1217, ctx-f951d8a4]) (logid:63df11b4) Hosts's actual total CPU: 12748 and CPU after applying overprovisioning: 12748
2026-09-07 11:14:24,717 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-63:[ctx-806840fb, job-1216/job-1217, ctx-f951d8a4]) (logid:63df11b4) Hosts's actual total CPU: 12748 and CPU after applying overprovisioning: 12748
2026-09-07 11:14:24,756 INFO  [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-63:[ctx-806840fb, job-1216/job-1217, ctx-f951d8a4]) (logid:63df11b4) Implementing network extension for network Network {"id": 227, "name": "test6", "uuid": "2e690dff-8d4e-4603-8454-7e06672bdc56", "networkofferingid": 40} (VLAN null)
2026-09-07 11:14:24,756 INFO  [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-63:[ctx-806840fb, job-1216/job-1217, ctx-f951d8a4]) (logid:63df11b4) Ensuring network device for network Network {"id": 227, "name": "test6", "uuid": "2e690dff-8d4e-4603-8454-7e06672bdc56", "networkofferingid": 40} (current={})
2026-09-07 11:14:25,306 DEBUG [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-63:[ctx-806840fb, job-1216/job-1217, ctx-f951d8a4]) (logid:63df11b4) Script output: {"host":"192.168.55.179","namespace":"cs-net-227"}
2026-09-07 11:14:25,316 INFO  [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-63:[ctx-806840fb, job-1216/job-1217, ctx-f951d8a4]) (logid:63df11b4) Ensuring network device for network Network {"id": 227, "name": "test6", "uuid": "2e690dff-8d4e-4603-8454-7e06672bdc56", "networkofferingid": 40} (current={"host":"192.168.55.179","namespace":"cs-net-227"})
2026-09-07 11:14:25,628 DEBUG [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-63:[ctx-806840fb, job-1216/job-1217, ctx-f951d8a4]) (logid:63df11b4) Script output: {"host":"192.168.55.179","namespace":"cs-net-227"}
2026-09-07 11:14:27,212 DEBUG [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-63:[ctx-806840fb, job-1216/job-1217, ctx-f951d8a4]) (logid:63df11b4) Script output: Cannot find device "vn--227"
2026-09-07 11:14:27,212 ERROR [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-63:[ctx-806840fb, job-1216/job-1217, ctx-f951d8a4]) (logid:63df11b4) Network extension script implement-network failed with exit code 3: Cannot find device "vn--227"
2026-09-07 11:14:27,261 DEBUG [c.c.n.f.FirewallManagerImpl] (Work-Job-Executor-63:[ctx-806840fb, job-1216/job-1217, ctx-f951d8a4]) (logid:63df11b4) applying default firewall egress rules
2026-09-07 11:14:27,263 INFO  [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-63:[ctx-806840fb, job-1216/job-1217, ctx-f951d8a4]) (logid:63df11b4) applyFWRules: network=Network {"id": 227, "name": "test6", "uuid": "2e690dff-8d4e-4603-8454-7e06672bdc56", "networkofferingid": 40} activeRules=0 defaultEgressAllow=true
2026-09-07 11:14:27,263 INFO  [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-63:[ctx-806840fb, job-1216/job-1217, ctx-f951d8a4]) (logid:63df11b4) Ensuring network device for network Network {"id": 227, "name": "test6", "uuid": "2e690dff-8d4e-4603-8454-7e06672bdc56", "networkofferingid": 40} (current={"host":"192.168.55.179","namespace":"cs-net-227"})
2026-09-07 11:14:27,565 DEBUG [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-63:[ctx-806840fb, job-1216/job-1217, ctx-f951d8a4]) (logid:63df11b4) Script output: {"host":"192.168.55.179","namespace":"cs-net-227"}
2026-09-07 11:14:29,149 DEBUG [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-63:[ctx-806840fb, job-1216/job-1217, ctx-f951d8a4]) (logid:63df11b4) Script output: apply-fw-rules: built 0 ingress rule(s) across 0 public IP(s), 0 egress rule(s), default_egress=ALLOW, default-DROP chains for 0 static-NAT IP(s) with no rules
2026-09-07 11:14:29,149 DEBUG [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-63:[ctx-806840fb, job-1216/job-1217, ctx-f951d8a4]) (logid:63df11b4) Ignoring non-JSON script output: apply-fw-rules: built 0 ingress rule(s) across 0 public IP(s), 0 egress rule(s), default_egress=ALLOW, default-DROP chains for 0 static-NAT IP(s) with no rules
2026-09-07 11:14:29,155 INFO  [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-63:[ctx-806840fb, job-1216/job-1217, ctx-f951d8a4]) (logid:63df11b4) Applying 1 IPs for network Network {"id": 227, "name": "test6", "uuid": "2e690dff-8d4e-4603-8454-7e06672bdc56", "networkofferingid": 40}
2026-09-07 11:14:29,156 INFO  [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-63:[ctx-806840fb, job-1216/job-1217, ctx-f951d8a4]) (logid:63df11b4) Ensuring network device for network Network {"id": 227, "name": "test6", "uuid": "2e690dff-8d4e-4603-8454-7e06672bdc56", "networkofferingid": 40} (current={"host":"192.168.55.179","namespace":"cs-net-227"})
2026-09-07 11:14:29,463 DEBUG [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-63:[ctx-806840fb, job-1216/job-1217, ctx-f951d8a4]) (logid:63df11b4) Script output: {"host":"192.168.55.179","namespace":"cs-net-227"}
2026-09-07 11:14:31,012 INFO  [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-63:[ctx-806840fb, job-1216/job-1217, ctx-f951d8a4]) (logid:63df11b4) Ensuring network device for network Network {"id": 227, "name": "test6", "uuid": "2e690dff-8d4e-4603-8454-7e06672bdc56", "networkofferingid": 40} (current={"host":"192.168.55.179","namespace":"cs-net-227"})
2026-09-07 11:14:31,319 DEBUG [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-63:[ctx-806840fb, job-1216/job-1217, ctx-f951d8a4]) (logid:63df11b4) Script output: {"host":"192.168.55.179","namespace":"cs-net-227"}
2026-09-07 11:14:33,117 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-63:[ctx-806840fb, job-1216/job-1217, ctx-f951d8a4]) (logid:63df11b4) Hosts's actual total CPU: 12748 and CPU after applying overprovisioning: 12748
2026-09-07 11:14:33,117 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-63:[ctx-806840fb, job-1216/job-1217, ctx-f951d8a4]) (logid:63df11b4) Hosts's actual total RAM: (6.57 GB) 7053959168 and RAM after applying overprovisioning: (6.57 GB) 7053959168
2026-09-07 11:14:34,682 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-64:[ctx-69cc0292, job-1216/job-1218, ctx-c4e49f32, FirstFitRoutingAllocator]) (logid:63df11b4) Hosts's actual total CPU: 12748 and CPU after applying overprovisioning: 12748
2026-09-07 11:14:34,709 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-64:[ctx-69cc0292, job-1216/job-1218, ctx-c4e49f32]) (logid:63df11b4) Hosts's actual total CPU: 12748 and CPU after applying overprovisioning: 12748
2026-09-07 11:14:34,711 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-64:[ctx-69cc0292, job-1216/job-1218, ctx-c4e49f32]) (logid:63df11b4) Hosts's actual total CPU: 12748 and CPU after applying overprovisioning: 12748
2026-09-07 11:14:34,711 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-64:[ctx-69cc0292, job-1216/job-1218, ctx-c4e49f32]) (logid:63df11b4) Hosts's actual total CPU: 12748 and CPU after applying overprovisioning: 12748
2026-09-07 11:14:34,751 INFO  [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-64:[ctx-69cc0292, job-1216/job-1218, ctx-c4e49f32]) (logid:63df11b4) Implementing network extension for network Network {"id": 227, "name": "test6", "uuid": "2e690dff-8d4e-4603-8454-7e06672bdc56", "networkofferingid": 40} (VLAN null)
2026-09-07 11:14:34,751 INFO  [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-64:[ctx-69cc0292, job-1216/job-1218, ctx-c4e49f32]) (logid:63df11b4) Ensuring network device for network Network {"id": 227, "name": "test6", "uuid": "2e690dff-8d4e-4603-8454-7e06672bdc56", "networkofferingid": 40} (current={})
2026-09-07 11:14:35,058 DEBUG [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-64:[ctx-69cc0292, job-1216/job-1218, ctx-c4e49f32]) (logid:63df11b4) Script output: {"host":"192.168.55.179","namespace":"cs-net-227"}
2026-09-07 11:14:35,084 INFO  [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-64:[ctx-69cc0292, job-1216/job-1218, ctx-c4e49f32]) (logid:63df11b4) Ensuring network device for network Network {"id": 227, "name": "test6", "uuid": "2e690dff-8d4e-4603-8454-7e06672bdc56", "networkofferingid": 40} (current={"host":"192.168.55.179","namespace":"cs-net-227"})
2026-09-07 11:14:35,391 DEBUG [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-64:[ctx-69cc0292, job-1216/job-1218, ctx-c4e49f32]) (logid:63df11b4) Script output: {"host":"192.168.55.179","namespace":"cs-net-227"}
2026-09-07 11:14:36,929 DEBUG [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-64:[ctx-69cc0292, job-1216/job-1218, ctx-c4e49f32]) (logid:63df11b4) Script output: Cannot find device "vn--227"
2026-09-07 11:14:36,929 ERROR [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-64:[ctx-69cc0292, job-1216/job-1218, ctx-c4e49f32]) (logid:63df11b4) Network extension script implement-network failed with exit code 3: Cannot find device "vn--227"
2026-09-07 11:14:36,978 DEBUG [c.c.n.f.FirewallManagerImpl] (Work-Job-Executor-64:[ctx-69cc0292, job-1216/job-1218, ctx-c4e49f32]) (logid:63df11b4) applying default firewall egress rules
2026-09-07 11:14:36,982 INFO  [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-64:[ctx-69cc0292, job-1216/job-1218, ctx-c4e49f32]) (logid:63df11b4) applyFWRules: network=Network {"id": 227, "name": "test6", "uuid": "2e690dff-8d4e-4603-8454-7e06672bdc56", "networkofferingid": 40} activeRules=0 defaultEgressAllow=true
2026-09-07 11:14:36,982 INFO  [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-64:[ctx-69cc0292, job-1216/job-1218, ctx-c4e49f32]) (logid:63df11b4) Ensuring network device for network Network {"id": 227, "name": "test6", "uuid": "2e690dff-8d4e-4603-8454-7e06672bdc56", "networkofferingid": 40} (current={"host":"192.168.55.179","namespace":"cs-net-227"})
2026-09-07 11:14:37,288 DEBUG [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-64:[ctx-69cc0292, job-1216/job-1218, ctx-c4e49f32]) (logid:63df11b4) Script output: {"host":"192.168.55.179","namespace":"cs-net-227"}
2026-09-07 11:14:38,864 DEBUG [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-64:[ctx-69cc0292, job-1216/job-1218, ctx-c4e49f32]) (logid:63df11b4) Script output: apply-fw-rules: built 0 ingress rule(s) across 0 public IP(s), 0 egress rule(s), default_egress=ALLOW, default-DROP chains for 0 static-NAT IP(s) with no rules
2026-09-07 11:14:38,864 DEBUG [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-64:[ctx-69cc0292, job-1216/job-1218, ctx-c4e49f32]) (logid:63df11b4) Ignoring non-JSON script output: apply-fw-rules: built 0 ingress rule(s) across 0 public IP(s), 0 egress rule(s), default_egress=ALLOW, default-DROP chains for 0 static-NAT IP(s) with no rules
2026-09-07 11:14:38,868 INFO  [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-64:[ctx-69cc0292, job-1216/job-1218, ctx-c4e49f32]) (logid:63df11b4) Applying 1 IPs for network Network {"id": 227, "name": "test6", "uuid": "2e690dff-8d4e-4603-8454-7e06672bdc56", "networkofferingid": 40}
2026-09-07 11:14:38,868 INFO  [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-64:[ctx-69cc0292, job-1216/job-1218, ctx-c4e49f32]) (logid:63df11b4) Ensuring network device for network Network {"id": 227, "name": "test6", "uuid": "2e690dff-8d4e-4603-8454-7e06672bdc56", "networkofferingid": 40} (current={"host":"192.168.55.179","namespace":"cs-net-227"})
2026-09-07 11:14:39,170 DEBUG [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-64:[ctx-69cc0292, job-1216/job-1218, ctx-c4e49f32]) (logid:63df11b4) Script output: {"host":"192.168.55.179","namespace":"cs-net-227"}
2026-09-07 11:14:40,738 INFO  [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-64:[ctx-69cc0292, job-1216/job-1218, ctx-c4e49f32]) (logid:63df11b4) Ensuring network device for network Network {"id": 227, "name": "test6", "uuid": "2e690dff-8d4e-4603-8454-7e06672bdc56", "networkofferingid": 40} (current={"host":"192.168.55.179","namespace":"cs-net-227"})
2026-09-07 11:14:41,046 DEBUG [o.a.c.f.e.n.NetworkExtensionElement] (Work-Job-Executor-64:[ctx-69cc0292, job-1216/job-1218, ctx-c4e49f32]) (logid:63df11b4) Script output: {"host":"192.168.55.179","namespace":"cs-net-227"}
2026-09-07 11:14:42,881 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-64:[ctx-69cc0292, job-1216/job-1218, ctx-c4e49f32]) (logid:63df11b4) Hosts's actual total CPU: 12748 and CPU after applying overprovisioning: 12748
2026-09-07 11:14:42,881 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-64:[ctx-69cc0292, job-1216/job-1218, ctx-c4e49f32]) (logid:63df11b4) Hosts's actual total RAM: (6.57 GB) 7053959168 and RAM after applying overprovisioning: (6.57 GB) 7053959168


*************************** 28. row ***************************
                            id: 227
                          name: test6
                          uuid: 2e690dff-8d4e-4603-8454-7e06672bdc56
                  display_text: test6
                  traffic_type: Guest
         broadcast_domain_type: Vlan
                 broadcast_uri: NULL
                       gateway: 10.1.1.1
                          cidr: 10.1.1.0/24
                          mode: Dhcp
           network_offering_id: 40
           physical_network_id: 200
                data_center_id: 1
                     guru_name: NetworkExtensionGuestNetworkGuru
                         state: Allocated
                       related: 227
                     domain_id: 1
                    account_id: 2
                          dns1: 8.8.8.8
                          dns2: NULL
                       ip6dns1: NULL
                       ip6dns2: NULL
                     guru_data: NULL
                    set_fields: 0
                      acl_type: Account
                network_domain: cs2cloud.internal
                reservation_id: f9fe4b5c-831e-4903-9258-7ab5338e27bd
                    guest_type: Isolated
              restart_required: 0
                       created: 2026-09-07 10:31:15
                       removed: NULL
             specify_ip_ranges: 0
                        vpc_id: NULL
                   ip6_gateway: NULL
                      ip6_cidr: NULL
                  network_cidr: NULL
               display_network: 1
                network_acl_id: NULL
                   streched_l2: 0
                     redundant: 0
                   external_id: NULL
                    public_mtu: 1500
                   private_mtu: 1500
keep_mac_address_on_public_nic: 1

What to do about it?

The vlan information of the guest network should be persisted in the database so that the vlan

Root cause found

NetworkExtensionGuestNetworkGuru.implement() (server/src/main/java/com/cloud/network/guru/NetworkExtensionGuestNetworkGuru.java:104-120):

public Network implement(final Network network, final NetworkOffering offering, final DeployDestination dest, final ReservationContext context)
        throws InsufficientVirtualNetworkCapacityException {
    ...
    // The broadcast_domain_type and broadcast_uri are set by the extension script output
    final NetworkVO implemented = getNetworkVO(network, offering, physicalNetworkId);
    return implemented;
}

private static NetworkVO getNetworkVO(Network network, NetworkOffering offering, Long physicalNetworkId) {
    final NetworkVO implemented = new NetworkVO(...);
    ...
    if (network.getBroadcastUri() != null) {
        implemented.setBroadcastUri(network.getBroadcastUri());   // only copies through — never allocates
    }
    return implemented;
}

Compare to the parent it extends, GuestNetworkGuru.implement() (server/src/main/java/com/cloud/network/guru/GuestNetworkGuru.java:398), which calls allocateVnet():

protected void allocateVnet(final Network network, final NetworkVO implemented, final long dcId, final long physicalNetworkId, final String reservationId)
        throws InsufficientVirtualNetworkCapacityException {
    if (network.getBroadcastUri() == null) {
        final String vnet = _dcDao.allocateVnet(dcId, physicalNetworkId, network.getAccountId(), reservationId, ...);
        implemented.setBroadcastUri(BUri(vnet));   // auto-allocates a fresh
VLAN
    } else {
        implemented.setBroadcastUri(n
    }
}

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions