Manufacturers are finding value deploying AGVs and AMRs to improve operational productivity and flexibility in the production environments. Logistics companies in many industries are investing in automated material handling systems with AGVs to improve operations and increase worker safety.
Click here to get more.
AGVs and AGRs move autonomously throughout production sites to independently fulfill various tasks or move products and parts through the system. This mobility is possible because of the advancement of reliable wireless communications.
AGVs are used to move product or components throughout a production facility. Typically an AGV solution comprises of a central control system, a navigation system, and multiple vehicles that move about the plant according to the needs of the operation. The vehicles are often customized to the needs of the operation.
AGVs communicate with the infrastructure about sensitive and time-critical information necessary for autonomous operation, such as receiving control commands. Any large network latency or long interruptions on the communication network can trigger AGV(s) to stop, leading to undesirable disruptions in the manufacturing process.
Autonomous Mobile Robots (AMRs) are currently being introduced in many intralogistics operations, like manufacturing, warehousing, cross-docks, terminals, and hospitals. Its advanced hardware and control software allow autonomous operations in dynamic environments. Compared to an Automated Guided Vehicle (AGV) system in which a central unit takes control of scheduling, routing, and dispatching decisions for all AGVs, AMRs can communicate and negotiate independently with other resources like machines and systems and decentralize the decision-making process. Decentralized decision-making allows the system to react dynamically to changes in the system state and environment.
In contrast to an AGV, an AMR navigates via maps that its software constructs on-site or via pre-loaded facility drawings. The AMR uses data from cameras and built-in sensors and laser scanners as well as sophisticated software that enables it to detect its surroundings and choose the most efficient route to reach its intended destination. It works completely autonomously and if forklifts, pallets, people, or other obstacles occur in front of it, the AMR will safely maneuver around them, using the best alternative route. This optimizes productivity by ensuring that material flow stays on schedule.
An AMR only needs simple software adjustments to change its missions, so the same robot can perform a variety of different tasks at different locations, automatically making adjustments to meet changing environments and production requirements. AMR tasks can be controlled via the robot interface or configured by fleet control software for multiple robots that automatically prioritizes orders and the robot that is best-suited for a given task based on position and availability. Once a mission is established, employees do not have to spend time coordinating the robots’ work, which allows them to focus on high-value work that contributes to company success.
The flexibility of AMRs is crucial for modern manufacturing environments that require agility and flexibility if there is a need for modifications to products or the production line. AMRs are highly adaptable for agile production in any size facility. If production cells are moved or new cells or processes are added, a new map of the building can be quickly and easily uploaded or the AMR can re-map onsite, so it can be used immediately for new tasks. This capability gives organizations full ownership of the robot and its functions.
Although an AMR consists of much more advanced technology than an AGV, it is typically a less-expensive solution. An AMR does not need wires, magnetic stripes, or other costly modifications to the building infrastructure so it is faster and less expensive to get AMRs up and running, and with no costly disruption to production in the process. Because AMRs can be deployed quickly and easily, they add new efficiencies almost immediately. With low initial costs and fast optimization of processes, they offer remarkably fast return on investment–often in less than six months
AGVs and AGRs rely upon wireless communication with local, centralized applications for safety, control, guidance and optimization. This communication has to be reliable and fast to keep the AGVs and AGRs operational. This section discusses some of the key communication requirements of these applications that the wireless network must support.
One of the most critical applications for AGVs and AGRs are the safety applications that keep the personnel, product, and AGVs/AGRs from harm. This section explains key considerations of the two most commonly used industrial safety protocols for controlling AGVs/AGRs: the ODVA Common Industrial Protocol (CIP) Safety and PROFINET Profi-Safe. A high-level overview of the protocols is provided along with the protocol characteristics and wireless design considerations to support the protocol.
The Common Industrial Protocol (CIP) allows users to integrate automation applications - including control, safety, synchronization, and motion - across all aspects of the business. CIP is a media independent protocol using a producer-consumer communication model, and is a strictly object oriented protocol at the upper layers.
Each CIP object has attributes (data), services (commands), connections, and behaviors (relationship between attribute values and services). CIP includes an extensive object library to support general purpose network communications, network services such as file transfer, and typical automation functions such as analog and digital input/output devices, HMI, motion control and position feedback.
A key feature of CIP is that it defines two types of communication, or messages: explicit and implicit. Explicit messages are used for “as-needed” data (information) and are transmitted via TCP (transmission control protocol). Implicit messages are used for control data (inputs and outputs) — where high speed and low latency are important — and are transmitted via UDP (user datagram protocol). The UDP protocol allows messages to be sent in smaller packet sizes and makes it possible to use the producer-consumer model for these critical, implicit messages.
This section provides an overview of the two most commonly used industrial/industrial safety protocols in use: the Common Industrial Protocol (CIP) and PROFINET. A high-level overview of the protocols is provided along with the protocol characteristics and wireless design considerations to support the protocol.
The Common Industrial Protocol (CIP) allows users to integrate automation applications - including control, safety, synchronization, and motion - across all aspects of the business. CIP is a media independent protocol using a producer-consumer communication model, and is a strictly object oriented protocol at the upper layers.
Each CIP object has attributes (data), services (commands), connections, and behaviors (relationship between attribute values and services). CIP includes an extensive object library to support general purpose network communications, network services such as file transfer, and typical automation functions such as analog and digital input/output devices, HMI, motion control and position feedback.
A key feature of CIP is that it defines two types of communication, or messages: explicit and implicit. Explicit messages are used for “as-needed” data (information) and are transmitted via TCP (transmission control protocol). Implicit messages are used for control data (inputs and outputs) — where high speed and low latency are important — and are transmitted via UDP (user datagram protocol). The UDP protocol allows messages to be sent in smaller packet sizes and makes it possible to use the producer-consumer model for these critical, implicit messages.
Figure 1 CIP Protocol Stack
CIP Safety is an extension to the standard capabilities of CIP, and it has been certified for use in functional safety applications. Allen Bradley systems utilize GuardLogix programmable safety controllers and safety remote IO modules. CIP Safety devices are connected to the same Ethernet/IP network as the other machine devices, utilizing the same network. The CIP device is responsible for confirming the integrity of the data, and if an error occurs, it will go into a safe state. There is also a CIP Safety application layer that validates the integrity of the safety data transfers between the safety PLC and the CIP Safety device. There are time stamps and production identifier information that are contained in each packet to ensure that the data is from the expected device as well as within the time expected.
Figure 2 CIP Safety Communication Mechanics
Figure 3 CIP Safety Application goes into Fail-Safe State if any one of the above failures occurs on the network
Figure 4 CIP Safety Application Layer Error Checking
To avoid the complexity and maintenance of designing a dedicated safety-rated network, IEC and IEC -3 emphasize another option called “the black channel”. The black channel assumes that network is completely unreliable, so diagnostics must exist outside of the network infrastructure. This concept stipulates that if a safety communication protocol has enough error detection built into the protocol, it can be transmitted independently across different network types without degrading the integrity of the safety data. This can include traversing multiple network links and network segmentation techniques.
Figure 5 Routing of CIP Safety Traffic across the network
Figure 6 CIP Safety Traffic Through Multiple Layers of an EtherNet/IP Network
Only the safety data that is needed is routed to the required cell, which reduces the individual bandwidth requirements. The combination of fast responding local safety cells and the inter-cell routing of safety data allows users to create significantly larger and more complex safety applications with fast response times.
CIP Safety devices create a logical connection to each other, independent of the network technologies being used. In the devices, common errors are mitigated with various techniques, as described in IEC -3-2. Time stamps are used with time expectation to detect if packets are lost, delayed, repeated, or transmitted out of order. Unique device identifiers are used to authenticate the communication between two safety devices. Additional diagnostics and checks are included to validate that the messages are not corrupted in transit and all these features are separate from standard communication methods.
When these mitigations are put together as CIP Safety, a single connection between two devices, wired or wireless, can be used for communications certified up to SIL 3 per IEC and up to Category 4/PLe per ISO -1.
CIP Safety over wireless is heavily leveraged in several applications, including factory automation in production environments like those found in automotive assembly plants. In these plants, several different types of conveyance systems, such as AGVs, are used to move vehicles and parts throughout the production cells. In these systems, the auto manufacturers prioritize the safety of their employees, and the flexibility of the system for future enhancements. This creates the requirement for Functional Safety via CIP Safety, and the wireless link to enable it.
In a typical AGV deployment, there are several AGVs traveling hundreds of feet. Each AGV is outfitted with electronics including sensors for collision avoidance, location awareness, and safety, as well as drives for propulsion, I/O, and a safety controller such as a Compact GuardLogix®. This AGV system requires a wireless communication connection to the main safety controller, which may be a GuardLogix®. This main controller also serves as the traffic cop for the entire AGV application and thus the system requires control, safety, and diagnostic data to be transmitted wirelessly.
PROFINET is the PROFIBUS International (PI) industrial Ethernet standard designed for automation control communication over Ethernet-based infrastructure.
PROFINET devices may require different communication speeds depending on the type of automation process. The PROFINET protocol supports three communication classes, each with a different degree of time sensitivity. These are Non-real-time (NRT), Real-time (RT), and Isochronous Real-time (IRT) communication.
NRT, sometimes referred to as TCP/IP communication, is acyclic traffic such as sensory, diagnostic, or maintenance data transferred at best-effort speed. RT communication is cyclic traffic consisting of high-performance process data transmitted over standard networking infrastructure. IRT communication is the highest performing type of deterministic traffic within the PROFINET standard. However, this requires hardware-based bandwidth reservation and network-wide clock synchronization to function.
The PROFINET RT and IRT communication classes involve a cyclic data exchange over standard Ethernet and take place directly on Layer 2 without any TCP/IP overhead to minimize latency. This means that in an RT/IRT PROFINET environment, data frames are forwarded based on the devices′ MAC address. Therefore, it is essential that any underlying network infrastructure deployed to support RT or IRT PROFINET applications is fully Layer 2 transparent to all connected PROFINET devices.
The performance of PROFINET-based communication is limited to the performance ceiling of the underlying network infrastructure. To provide the flexibility to operate reliably over the different network infrastructure components, the cyclic data exchange rate for PROFINET RT communication can be customized to accommodate any infrastructure limitations or to suit the automation context.
In the example using the Siemens TIA Portal, the IO cycle > Update time parameter defines the communication update interval between the PROFINET IO controller and the IO devices. The IO cycle > Watchdog time parameter specifies the number of consecutive response failures before reporting a link failure which, depending on the process design, typically triggers the error handling or safe mode, halting the automation process.
PROFIsafe is an additional software layer that provides functional safety over the bus in PROFINET (or PROFIBUS) networks. PROFIsafe will take care of the functional safety portion of communications. It ensures the integrity of failsafe signals transmitted between safety devices and a safety controller meeting the relevant safety standards for industrial networks (up to SIL3 according to IEC / IEC , or Category 4 according to EN 954-1, or PL “e” according to ISO -1).
Figure 7 PROFISAFE Protocol Stack
The diagram above illustrates the difference between PROFIsafe and PROFINET in terms of roles. PROFINET is the overall communication protocol. PROFIsafe is an additional layer that fulfills the functional safety requirements of the application.
PROFIsafe ensures the integrity of failsafe signals transmitted between safety devices and a safety controller meeting the relevant safety standards for Industrial networks. It is even approved for wireless transmission channels such as WLAN and Bluetooth. Therefore, PROFINET users can utilize PROFIsafe for functional safety and transmit safety messages over their standard network, which can include WLAN or Bluetooth connections.
The internal transport systems for routing and supervising AGVs often use navigation systems. However, precise positioning is required at all times, for e.g., an AGV that is moving in an environment with other AGVs or other moving elements such as people. AGV should not only avoid obstacles but do it optimally. Avoiding means not only stopping but trying to predict them and find an alternative, safe route. Using all the possible sensors and communication devices all the time is not very energy efficient. It must be emphasized that most AGV platforms have limited battery capability – one of the goals is to increase the operating time of an AGV before it needs to be docked for a recharge.
Communication must be well planned according to the requirements of the industrial environment. Time-critical processes are an innate characteristic of several industrial processes. This time parameter imposes a strict delay requirement for the communication (deterministic). There are some elements that should be considered.
■ Reliability (fault-tolerance): The goal of reliability is to ensure that an AGV reaches its destination, and in the case of serious loss of communication, an emergency routine must be activated.
■ Security: Self-configuration and automation have potential vulnerabilities for attackers to exploit and take control of a system. A harsh environment, unpredictable variations in interference and interruptions, reflections from walls and floors, noise generated from equipment and machinery, etc.
■ Availability (redundancy required, what happens if crucial data is lost, how to prevent crucial data from being lost). Heterogeneous devices with different energy resources, a CPU, and memory must communicate efficiently. The goal is that the moving nodes maintain continuous network connection as it leaves and enters new areas of the network topology. For the network layer to support this roaming, redundant paths are probably part of the solutions. Redundancy among the paths, specifically, redundancy among the successor nodes, means that the area in which a node can move while still being within reach of at least one of the current successors is increased. Lost successors must pre-emptively be exchanged by new successors to maintain network connectivity. For the MAC layer to support such roaming, the moving nodes must be continuously included in the schedule of the new areas.
The tremendous growth in automated material handling systems is a sure sign that these systems deliver on the promise of improved operations. Still, system reliability is critical to realizing the value of an investment in an AGV system. AGV vendors and system integrators around the world have turned to technology to address one critical element of the system - wireless communications between the central control system and the vehicles.
Many AGV applications need to transmit I/O signals from each vehicle. I/O traffic does not require a lot of bandwidth, but low latency and uninterrupted communication is critical. Even a short 50 to 70 mSec disruption in the link can cause the I/O system to fault, stopping the AGV in its tracks.
As AGVs move about the plant, their wireless links must "roam" from one fixed radio to another - and these roam events can cause enough delay to trip the I/O system offline.
Another factor for wireless communication reliability in AGV applications is the dynamic nature of the environment. Plants are constantly changing, with people, parts, and machinery moving around, blocking radio paths, reflecting or absorbing radio signals, and generally making things difficult for the AGVs to find an infrastructure radio with a good quality RF signal for connection.
■ Strict Low Latency (implies lower contention and packet retries over wireless medium)
■ AS/RS equipment are tasked to move and fetch goods quickly and efficiently. This is enabled by seamless wireless roaming. Roaming/handoffs enables a mobile wireless client to search for and associate with an access point with the best signal before the signal strength from the currently connected access point significantly degrades as the vehicle moves away from it or due to any obstructions or interference. As the AS/RS systems are always mobile, AGV/AMR automation and control processes rely on stable and highly responsive networks. Achieving millisecond-level wireless roaming/handover times therefore becomes a necessity to minimize latency and avoid impact to operations.
■ Support for CIP and PROFINET industrial protocols. Wired PLC(s) need to communicate with PLC on-board the AGV/AMR to send control commands.
■ QoS for prioritizing real-time applications over all other kinds of traffic traversing the wired and wireless network
■ There is a lot of RF noise in industrial environments, the same kind of radio frequencies wireless networks run on.
■ There is also usually heat, dust, and other contaminants on the floor that require more hardy and ruggedized wireless networking equipment than the typical indoor enterprise APs.
■ The temperature specs, the vibration specs, and metal enclosure versus plastics all need to be considered. Industrial hardware has to be equipped for heat, dust, and dirt.
When compared to wired networks, wireless provides a set of challenges when there is a need to support Real-time applications:
■ RF conditions change drastically and this is especially true in the case of mobile clients
■ Handoff between access-points can be detrimental to real-time applications
■ Radio waves from the radio transmitting devices lose strength exponentially as they propagate away from the transmitter. Even when two devices are physically close to each other, if their transmitting equipment and receiving equipment are focused for a narrow transmission field but not aligned to each other, the communications could be transmitted without being received. Similarly, obstacles and other materials in the environment can block the signal or weaken it.
■ Other wireless devices in proximity of the network and operating within the same frequency band are capable of causing interference on the network.
■ Obstructions or interferences entails that the RF environment can change rapidly. As signal strength weakens, wireless devices will hop to another access point, which can result in additional latency or packet loss.
The most important quality indicators of how a wireless network can meet the requirements for AGV/AMR applications are:
■ Packet loss rate: the percentage of sent messages (or packets/frames) that are not successfully received by the intended recipient.
■ Latency: the delay in transmission for the delivery of a message via a wireless connection.
■ Jitter: variation in Latency.
■ Data throughput of the wireless connection: the ability to transmit a certain amount of data within a specified time interval. This is not a major concern for AGV/AMR applications since they are low throughput applications.
■ Handoff Time - Interruption: a break in transmission that takes place when a wireless client roams from one infrastructure access point to another.
■ Wireless Coverage Area: the area covered by an access point or the seamlessness in the coverage of a facility that determines whether the wireless signal is strong enough to reach everywhere along the AGV/AMR traversal path.
However even with the challenges listed above, with certain advancements in wireless technology over the years, performing a proper site-survey, RF planning and design, end-to-end QoS prioritizing real-time traffic over all other traffic, and optimized roaming between access points (CURWB supports a 0mSec handoff time as will be seen later) we are now able to support most of the real-time application needs which includes industrial applications such as AGVs, AMRs, PLC to PLC, and PLC to IO communications over wireless.
Designing and deploying a robust wireless network that can keep up in an industrial environment requires forethought, careful design, and expert help. Assuming responsible administration, careful planning and the availability of trained employees who are aware of the specific concerns pertaining to wireless networks, they are as reliable, secure and robust as wired networks.
Wireless is the only practical medium that can be leveraged for applications like AGV/AMR.
This section starts by providing an overview of the wired and wireless network components needed to deploy the solution, the architecture to support AGVs/AMRs in a Factory/Warehouse environment, followed by some design best-practices around High Availability, QoS and Security.
All Fluidity / fixed infrastructure deployments need a mesh end. It functions as a gateway between wireless and wired. It is highly recommended that all systems using Fluidity use a redundant pair of mesh end gateways to terminate the MPLS tunnels, aggregate traffic and act as an interface between the wired and wireless network. Mesh End gateways can also be thought of as MPLS label edge routers (LERs) on the infrastructure network. The Mesh End gateway is responsible for encapsulating the traffic coming in from the wired network into the Fluidity overlay network using MPLS and de-encapsulating MPLS and delivering standard datagrams onto the wired network.
CURWB gateways are also used in the role of Global Gateways within a L3 Fluidity deployment where-in they form an L2TP tunnel to each of the Mesh Ends of the different subnet clusters.
It is useful to understand that the CURWB Gateways do not run any RF or handoff logic and are not to be considered as Wireless Controllers within a traditional Wi-Fi deployment.
CURWB gateways are rugged, industrial grade network appliances that make setup and management of medium and large-scale CURWB Fluidity and Fixed Infrastructure deployments fast and easy. Gateways allow the Fluidity wireless infrastructure to scale to hundreds of radio devices, without impacting the performance of the overall network.
Figure 14 FM and FM Mesh End Gateway Devices
Table 1 CURWB Gateway Models comparison
Figure 15 FM and FM Fiber Radio Unit
The FM MOBI comes in a rugged die cast aluminum housing that has been purpose built for harsh industrial environments. It consists of industrial-grade anti-vibration M12 ports and QMA connectors, EN certified. Optionally, you can also order the fiber-enabled FM MOBI which supports a fiber port with an XCO connector.
The Ethernet model has 2 x 10/100/ M12 ports. The Fiber model has 1 x Dual LC ruggedized SFP XCO connector (transceiver not included) and 1 x 10/100/ M12 port. The radio can either be powered using PoE+ output from a switch or 48V DC input from an onboard power source.
Note: It is highly recommended to have a DC-to-DC converter on board vehicles to provide stable/clean power at the appropriate voltage level to the radio to avoid any damage. This is applicable when powering on the radio using the vehicle battery.
The FM MOBI is the recommended radio model to be deployed on board the AGV/AMR, since it is vibration resistant. The FM MOBI can also be used as the infrastructure radio.
Figure 16 FM Endo
The FM Endo can be exploited as a wayside radio in any vertical market that features or depends on mobility applications. As an alternative to the FM radio the FM can also be leveraged as a wayside radio within the Factory AGV/AMR deployment. The FM Endo is a 2x2 MIMO radio with 2 x 10/100/ RJ45 interfaces.
CURWB RACER is a centralized cloud-hosted server that can be used for provisioning of the entire CURWB system including configuration, firmware upgrade, and plug-in activation. It allows all the radio configuration to be done in a single pane and uploaded to the radios in real time or offline. RACER supports almost all the configuration options (basic and advanced).
RACER can be used to create configuration templates, fill in the template with the required parameter values to create radio configurations, and apply them to multiple CURWB devices of the same type. Configurations created in RACER can be applied to the radio in either online mode (if the CURWB devices have Internet access) or offline mode (if the CURWB devices have no Internet access). The advantage of using RACER is that along with the device configuration it also upgrades the firmware to the latest version available and also applies the configured plug-ins. This is the preferred method for configuring CURWB devices for any size deployment.
Figure 17 CURWB RACER
FM-Monitor is a network-wide, on-premises monitoring dashboard, allowing any CURWB customer to proactively maintain and monitor one or multiple wireless OT networks. FM-Monitor displays data and situational alerts from every CURWB device in a network, in real time.
FM-Monitor supports fixed and Fluidity network architectures and allows easier end-to-end troubleshooting. It can be operated as a standalone system or in parallel with a sitewide Simple Network Management Protocol (SNMP) monitoring tool. It is designed to support network installations used in smart cities, rail, mining, ports and terminals, entertainment, smart factories, and military applications.
Figure 18 FM-Monitor Topology View
Features and benefits:
■ On-premises monitoring tool for CURWB networks
■ Wizard setup for quick and easy installation and deployment of FM-Monitor
■ Real-time dashboard displaying uptime, throughput, latency, jitter, and other network KPIs
■ Customizable section view to easily check groups of radios
■ Customizable monitoring alerts for prompt response
■ Radio-by-radio data logging with a minimum sampling interval of 300 mSec
■ Real-time information display for quick and accurate troubleshooting
■ Side-by-side comparison of radio KPIs over time and over vehicle position
■ Alerts/events can be forwarded to a Syslog server
■ Radio KPIs such as RSSI, LER, PER, etc. can be exported to a CSV file for graphing
The dashboard shows overall network performance and offers customizable segmentation of the network into clusters. This allows for easy monitoring of network sections or parts of a fleet of vehicles, maximizing network usage and performance. Clusters can include backhaul point-to-point links, point-to-multipoint distribution networks, vehicle access networks, wayside networks, and vehicle-mounted radios. FM Monitor displays and tracks real-time Key Performance Indicators (KPIs) within each cluster, including the number of active radios, number of connected IP edge devices, end-to-end latency, jitter, upload/download throughput in real time, and system uptime.
Figure 19 FM-MONITOR Dashboard
The table view allows customers to condense sections of the network into a tabular view, isolating specific radio configurations and performance statistics. During troubleshooting, this drastically reduces the time needed to understand system performance on a radio-by-radio basis.
Figure 20 FM-Monitor Table View
Figure 21 FM-Monitor Topology View
The following section describes some prerequisites to understand the CURWB architecture and deployment.
Figure 22 Mesh-ID
The Mesh-ID is a hardware identifier for the CURWB Gateways and Radios. It is pre-programmed from the factory with a hard-coded value which cannot be modified. It follows the Format of 5.x.x.x. Note that this is NOT an IP Address. The Mesh-ID is a decimal representation of the MAC address of the wireless interface. So if one were to convert the Mesh ID (from decimal to a Hex value) we will get back the MAC address of the wireless interface. That is how we guarantee the uniqueness of the Mesh-ID.
The Mesh-ID is relevant within the constructs of network design. A gateway/radio with lower Mesh-ID becomes the “primary”. Also the gateway/radio with the lowest Mesh-ID becomes the Mesh-End (if one is not explicitly configured).
The Mesh-ID of the radios is also used for license activation and receiving its configuration from RACER. RACER outputs a single file containing the licenses and configuration for a group of similar radios. The way the appropriate licenses and configuration is applied to a particular radio is based on its Mesh-ID. The Mesh-ID is tied to the Serial Number.
Figure 23 Passphrases
CURWB gateways/radios are configured with shared passphrases. CURWB control plane traffic is encrypted using this passphrase. The passphrase can also be used a means to segment a particular network in that radios with the same shared passphrase form a cluster and are kept separate from other CURWB MPLS networks which use a different passphrase.
Note: Data-plane / user traffic is not encrypted using the passphrase. In order to encrypt data-plan / user traffic, AES encryption must be enabled on the gateways/radios.
Note: If a shared passphrase is defined, the same passphrase must be used for all CURWB units in the same network. As a deployment best-practice configure the passphrase to be something other than the default value of “Fluidmesh”. The shared passphrase can be composed of any ASCII characters except the following: '`"\$=
A logical Mesh End (ME) can be redundant consisting of two physical Mesh End gateways/radios with the TITAN high-availability plug-in. The ME is typically configured within the Distribution or Core network. The purpose of the Mesh End radio is to terminate all the MPLS label-switched paths and act as a gateway between the CURWB network and the wired network. The ME holds all the Label Switch Paths (LSPs) to all the other radios in its database.
Note: Even though the CURWB solution has the capability to automatically select the gateway/radio with the lowest Mesh-ID to become the ME, as a best-practice it is highly recommended to configure the role of Mesh End and Mesh Point(s) manually within the deployment to have more deterministic convergence in case of failure within the network.
■ Similar considerations as for normal MPLS
■ MTU at endpoint =
■ Minimum required MTU on switches =
■ Radios don’t have to be configured with MTU – this is done automatically
The reason we need to increase the MTU size from the default value of to is to account for the addition of the MPLS header and avoid any fragmentation on the wired network.
STP is a Layer 2 protocol that runs on switches to prevent loops in the network when there are redundant paths in the network. Switches run the STP algorithm when they are first connected to a network or whenever there is a topology change. CURWB radios do not participate in the STP alongside the switches. The radios simply forward or block the BPDU messaged depending on how they are configured. CURWB radios have an equivalent process to STP, called AutoTap, and this helps avoid any loops within the wireless network.
BPDU Snooping can be enabled or disabled on the radio, according to the configuration the radio will act or not act on the contents of the BPDU.
BPDU forwarding, configured as ‘Pass’, forwards all the BPDUs. BPDU forwarding, configured as ‘Auto’, forwards the BPDUs in the wayside space and not forward them to the vehicle space and vice-versa. When BPDU forwarding is configured as ‘Stop’, no BPDUs are forwarded.
AutoTap is a network loop prevention mechanism that allows CURWB radios to detect connections and allow only a dedicated ingress/egress route to and from the Mesh End or network core.
With AutoTap, only one radio will publish MAC address information, and traffic is seen coming from only one radio that gets elected as the Primary radio of the physically connected redundant group. The radio with the lowest Mesh ID is selected as the Primary radio which advertises its MAC address. With this configuration, the radios are able to detect each other over the wired connection, and forward traffic to the connected radio utilizing this connection. Routes to the core and end devices are built automatically. The result is like having a single radio with multiple wireless interfaces.
AutoTap open on the Gateway (Core Network) represents the Label Edge Router (LER). It is the unit responsible for the MPLS encapsulation/de-encapsulation of traffic from/to the wired network. All other CURWB devices within the network (secondary gateway and wayside radios) act as MPLS Label Switch Routers (LSRs). They only forward the traffic based on the Labels. On a Vehicle with more than one radio connected to the same broadcast domain, again one of the two radios (the radio with the lower mesh ID) will have the role of a LER and the other radio (with the higher mesh ID) will have the role of LSR. The LER will have its AutoTap open and the other radio on the vehicle will have its AutoTap closed.
As a best practice, it is highly recommended that NTP be configured on the CURWB radios. A primary and secondary NTP server IP can be configured during RACER template configuration. When enabling NTP on the radio, it will synchronize time from the NTP server usually within an hour, however we can force it to happen sooner, the connection will be down for milli-seconds when forcing the radio to connect to the NTP server.
Large network optimization (LNO) is useful in large network environments of more than 50 infrastructure radios where it helps optimize the MPLS forwarding table by only establishing LSPs toward the Mesh-End unit.
The Mesh-End is the ingress/egress point of the MPLS domain. Spanning Tree Protocol (STP) is also affected in that BPDU forwarding will be disabled.
If LNO is enabled, the Mesh points will only establish LSPs with other Mesh-End devices, and it also disables STP packet and BPDU forwarding.
If LNO is disabled, LSPs will be created between all Mesh-points, and between Mesh points and Mesh-ends. STP packets and BPDU forwarding will be set to Automatic.
By default LNO is enabled.
The Fluidity rate adaptation setting controls the unit’s choice of modulation coding and speed of packet transmission. Fluidity supports two different rate adaptation modes:
■ Standard: This option applies a standard re-active rate selection as used by Wi-Fi technology
■ Advanced: This option applies CURWB proprietary predictive rate selection algorithm.
For the AGV/AMR use-case it is recommended to leverage the Advanced rate adaptation. The CURWB predictive rate selection algorithm is pro-active. When the link error rate (LER) increases there is no packet drop as the LER is kept low by predictively adjusting the data rate. The predictive algorithm tries to keep the LER and packet loss low by selecting a more conservative data rate. As opposed to this, the standard rate selection algorithm would need higher LER and packet drops to adjust to a lower data rate. Rate selection is also important to obtain good performance and to maximize the throughput of the radio communication system.
The RSSI prediction is performed by the transmitting radio using explicit feedback received from the destination radios. This results in a good estimate of the upstream channel condition. For further accuracy the system also filters out all instantaneous variation that may have a detrimental impact on the choice of the transmission rate.
The transmission rate is then selected according to the prediction of the estimated drawing from a small set of optimal rates computed by a heuristic channel estimation algorithm. Within high-speed mobility environments, the channel state changes very quickly. Therefore, it is important that the rate sampling algorithm finds the optimal rates in the shortest time possible while keeping accuracy. Failure to meet either condition typically results in low throughput, high latency, and high packet error rate (PER).
When the rate adaptation is set to Advanced, the vehicle radio bases the selection of MCS based on the RSSI received from the wayside infrastructure radio. The RSSI scale is divided into a number of non-overlapping bins, each bin corresponding to a subset of MCS values to be sampled by the rate selection algorithm. The RSSI bin definitions and their associated MCS subsets can be determined according to several criteria, which may include, for example, the sensitivity thresholds of the underlying wireless hardware. The table below shows the default values for the RSSI bins and the corresponding MCS rate selected.
Table 2 MCS Rate Selection based on RSSI bins (default values)
MCS Rate
(20-MHz wide channel)
Default RSSI Bins Min Max-95 : -77
0
2
-77 : -71
1
3
-71 : -67
2
4
-67 : -59
3
5
-59 : -49
4
6
-49>
5
7
The CURWB solution provides certain advanced handoff parameters for vehicle radio units that can be tuned depending on the RF environment to achieve optimal handoffs.
The RSSI zone threshold and Handoff hysteresis threshold features provide safeguards against unwanted handoffs – in other words, against unreasonably long periods of time between received signal strength from the connected unit falling too low, and a handoff request from a relief unit.
The relationship between these three settings governs whether a handoff will take place from one unit to another, based on a difference in comparative signal amplitude values over a period of time.
The RSSI low/high zone threshold sets the border between the low and high RSSI zones. In this case, as represented by the two graphs below, the -60 dB level marks the border between the low and high RSSI zones.
The threshold value is always expressed as SNR, with -95 dBm as the reference value, and is always expressed as a value greater than 0. The default value is 35. This equates to -60 dBm.
The default Fluidity delta-threshold is -60dBm. The default delta-high threshold is 6 dBm. What this means is that in good RF environments where the signal strength is higher than -60 dBm, the vehicle radio will only attempt a handoff to another wayside infrastructure radio if it provides a signal that is at least 6 dBm higher than what it is receiving from its currently connected wayside infrastructure radio. If the delta value is lower than 6, no handoff will occur at that time.
Figure 24 Fluidity delta-high example
The default delta-low threshold is 3 dBm. What this means is that in poor RF environments where the signal strength is lower than -60 dBm, the vehicle radio will attempt a handoff to another wayside infrastructure radio if it provides a signal that is at least 3 dBm higher than what it is receiving from its currently connected wayside infrastructure radio. If the delta value is lower than 3, no handoff will occur at that time.
Figure 25 Fluidity delta-low example
Note: The Fluidity delta-threshold, the delta-high and delta-low values are all configurable to values different from the default using either RACER or the radio CLI if needed for your RF environment tuning.
Figure 26 RACER Fluidity Advanced Handoff parameters
Below describes the typical behavior of a CURWB radio:
■ Populate local VBR (Virtual Bridge Routing) table with local end points
If you are looking for more details, kindly visit Daulfinld.
■ Over each antenna (with configured channel/frequency): Find peer radios
– Condition: Same passphrase, same cluster-id
Over LAN interface:
■ Find peer radios and gateways
■ Condition: Same passphrase
Prodigy - Build LSPs:
■ Pseudowire-set: “Mesh-End only”: To Mesh End only
■ Pseudowire-set: “All”: To all other radios
■ Radio metrics determine path: Higher signal strength wins over alternative path (over full path)
AutoTAP:
■ Used for Loop Avoidance when two radios using the same passphrase are directly connected using the wired and wireless network
■ Directly connected radios select a primary; radio with lower mesh-id becomes primary; only primary announces local endpoints either connected via switch or directly to a radio
■ The AutoTAP feature kicks in automatically and no configuration is needed
■ For each LSP: announce local VBR table, Learn remote VBR table entries
Note: AutoTAP is independent of LSPs. It does not influence LSPs or block switch ports (like STP does). It does however block the Ethernet interface of the CURWB radio if a loop is detected. From the directly connected switche’s perspective the port will still show as up and running, however in actuality the port will be blocked by the CURWB radio’s AutoTap logic.
The CURWB Prodigy 2.0 forwarding engine embeds a DiffServ-inspired framework to provide end-to-end QoS treatment to user traffic. The system support 8 priority levels, numbered from 0 to 7 with the former being the lowest and the latter being the highest, respectively.
When a client device IP packet first enters the mesh at an ingress CURWB router, the TOS/DSCP field of the IP header is inspected in order to assign a priority class to the packet. The class number is taken from the 3 most significant bits (b7-b5) of the TOS/DSCP field.
The default system setting is to preserve the original priority marking unchanged. The priority tag is then preserved through the whole end-to-end path to the egress router, where the packet leaves the CURWB network to be delivered to the destination client device. Priority scheduling is applied at the different transmission interfaces for each hop along the path.
For packets being transmitted over the wireless, the eight priority levels are further mapped into four access categories after scheduling (see table below). Each access category corresponds to a specific set of MAC transmission parameters which provide different levels of robustness and performance.
Table 3 Mapping between Packet Priority and Access Category
Priority
Access Category
0
Background (BK)
1
Background (BK)
2
Best Effort (BE)
3
Best Effort (BE)
4
Video (VI)
5
Video (VI)
6
Voice (VO)
7
Voice (VO)
An important thing to note here is that for traffic marked with CoS-6 and 7, CURWB disables packet aggregation which can otherwise delay the packet transmission over the air and add to end to end latency which is not desirable for real-time AGV control traffic.
Figure 27 MPLS EXP bits
■ The MPLS experimental 3 bits (EXP) are used to provide QoS capabilities in MPLS environments
■ The 3 EXP bits allows 8 different classifications of traffic, based on information found in the Layer 3 DSCP, or the Layer 2 CoS
■ During the MPLS encapsulation process, the QoS markings are copied into the EXP field in the MPLS header from the IP header, or from the ethernet header, which is the PCP field in the VLAN tag
■ For QoS purposes, CURWB units do not examine the IP header or ethernet header during the forwarding process. They examine the EXP bits, giving prioritization of one traffic type over another based on the assigned markings.
■ The MPLS experimental 3 bits is highlighted in yellow in the MPLS Header shown in the figure above
All client traffic within the MPLS tunnel is kept private (not encrypted) using the system passphrase, however for additional security the CURWB solution also supports enabling AES encryption to encrypt all traffic over the wireless medium. It is highly recommended to enable AES encryption within an AGV/AMR deployment.
Note: To enable AES encryption feature, an AES plug-in needs to be installed on the radio. When configuring AES encryption, it is mandatory to enable AES encryption on all the radios within the system. Enabling AES only for a sub-section of the system is not supported and will cause a breakage. The FM/ gateways already don’t need an AES plug-in.
L2 Fluidity provides simple and seamless mobility within a single broadcast domain. All the devices (PLCs, I/O, CURWB radios, CURWB gateways) use the same subnet and default gateway. Full VLAN support is available. The disadvantage of L2 Fluidity is that it is only constrained to a single broadcast domain and provides limited scale with a maximum of ~ 200 devices.
In case of a L2 Fluidity deployment no router is needed on-board the vehicles since the entire deployment is a flat L2 network. From each vehicle radio there is an MPLS Label Switched Path (LSP) to each Mesh End. In case or redundant Mesh Ends and redundant radios on the vehicle, there will be a total of 4 LSPs per vehicle and only one of the LSPs will be active. Any IP traffic received from the on-board devices (PLC, I/O, etc.) received by the onboard radio(s) is encapsulated into MPLS packet and sent across an LSP towards the CURWB Mesh End. For small deployments the CURWB mesh can use a CURWB radio may be able to be used as a Mesh End. For larger deployments a CURWB gateway like the FM should can be used. The Mesh End receives the MPLS packet, de-encapsulates it and delivers the original IP packet to the distribution/core network.
Figure 28 L2 Fluidity Packet Flow
The figure above depicts the L2 Fluidity packet flow from the vehicle to the control room. The laptop on the vehicle is configured to be in Access VLAN 222 and the PLC is configured to be in Access VLAN 220. The packets from the laptop and PLC on board the vehicle are untagged when they enter the on board switch. Assuming an IPv4 payload of bytes, the untagged ethernet frame size is bytes. The switch on board the vehicle adds the appropriate IEEE 802.1q VLAN tag to each of the packets. After the IEEE 802.1q VLAN tag of 4-bytes is added, the frame size is now bytes. The IEEE 802.1q tagged packets are then forwarded to the CURWB radio on board the vehicle.
The CURWB radio on board the vehicle then takes those packets and inserts an MPLS label, tags them with the CURWB Mgmt VLAN IEEE 802.1q tag (VLAN-248) and forwards them to the wayside infrastructure CURWB radio to where the vehicle radio is currently connected. Along with the MPLS header of 8-bytes an additional IEEE 802.1q VLAN tag of 4 bytes and an ethernet frame of 14-bytes the frame size now becomes bytes. The MPLS encapsulated frames with an IEEE 802.1q tag of VLAN-248 are forwarded to the CURWB FM gateway.
The CURWB FM gateway de-encapsulates the MPLS header and forwards the original IEEE 802.1q tagged frames onward to the control room switch. The control room switch removes the 802.1q header and forwards the frames towards their intended destination.
L3 Fluidity provides seamless mobility with higher scalability. In case of a L3 Fluidity deployment each wayside section has its own subnet and each vehicle has its own subnet. L2TP tunnels link each of the wayside sections to the Global Gateway(s). A L3 Fluidity deployment is slightly more complex as compared to a L2 Fluidity deployment. As in L2 Fluidity, the MPLS LSP for each of the vehicles terminates at the Mesh End. The packet is then forwarded into an L2TP tunnel and forwarded towards the Global Gateway which de-encapsulates the L2TP header and forwards the original IP packet to the core network. Any traffic entering the CURWB network from the core network is encapsulated within the L2TP tunnel and forwarded to the appropriate Mesh End.
In case of a CURWB single-frequency design, all the CURWB radios infrastructure and vehicles operate on the same frequency which is statically configured. An advantage of this design is that the vehicle radios don’t need to perform any off-channel scanning and hence enable seamless 0 mSec handoff from one infrastructure AP to another. Another big advantage of this design is that the deployment can use the cleanest channel available (determined as an outcome of the site spectrum analysis) and interference on other neighboring channel does not affect the system hence providing reliable wireless connectivity. Since the throughput and density requirements for a typical AGV/AMR system is not large a single-frequency design should be able sufficient to satisfy all requirements.
For large AGV/AMR deployments with a high density of AGV/AMR per infrastructure AP a single frequency design might most likely result in high channel utilization leading to higher contention on the wireless medium resulting in higher link and packet error rate. This is not desirable for a real-time control application such as AGV/AMR.
An alternative to the single frequency design described above CURWB also supports a Multi-Freq design to address the needs of larger and denser vehicle deployments. In a multi-frequency design two or more frequencies can be selected to provide coverage across the plant floor. For large and dense AGV/AMR deployments this design will provide superior performance by helping reduce the channel utilization (resulting in lower contention, retries and packet loss) and thus enabling a more stable and reliable wireless deployment.
CURWB supports two designs for implementing a multi-frequency architecture – Static and Dynamic.
In order to implement a Static Multi-Frequency design we need to increase the number of wayside infrastructure APs. The APs should be deployed in such a manner that coverage for all frequencies should be available throughout the plant floor. In this design a subset of vehicles will be operating on Freq-A and a subset of vehicles will be operating on Freq-B as an example. The design needs to ensure that there is appropriate coverage for both Frequencies A and B across the plant floor. For the static multi-frequency design there is no change needed on the vehicle side, with each of the vehicles only needing a single CURWB radio like in the single frequency design.
The dynamic multi-frequency design is pretty similar to traditional Wi-Fi design where-in adjacent infrastructure APs is on a different frequency (Freq-A, Freq-B, Freq-C, Freq-A, Freq-B, Freq-C and so on).
In order to implement a Dynamic Multi-Frequency design a change needs to be made on the vehicle end. Due to the nature of the design, the vehicle radio now needs to go off-channel to look for an AP providing a better signal while roaming across the plant floor. This functionality is termed as off-channel scanning. In order to perform off-channel scanning, the vehicle AP servicing client traffic on the current channel briefly needs to stop servicing client traffic and jump off-channel to scan for a better infrastructure AP to associate with. This results in a brief interruption to the client traffic which is not acceptable for real-time applications such as AGV/AMR control applications. To overcome this challenge the CURWB dynamic multi-frequency design supports installing two APs on each of the vehicles with one AP actively servicing client traffic while the second AP can perform the dedicated off-channel scanning functionality. This dual-AP vehicle design provides the same 0 mSec seamless roaming available in the single-frequency design.
■ AGV/AMR density within a particular region of the Warehouse or Factory Plant Floor. Total number of AGV/AMR that need to be supported at a given site.
■ Support for any other type of traffic over the wireless network other than Industrial protocol (e.g., unicast, multicast, broadcast, GPS data, etc.)
■ Provide good radio coverage across the intended AGV/AMR path.
■ High vehicle densities result in more interference and contention in the affected areas. The OMNI-3, an antenna designed for optimum performance at low signal strengths, is most suitable for this purpose.
■ Factory/Warehouse environments will most likely have metallic objects which could cause signal blockages and unwanted signal reflections.
■ An on-site spectrum analysis must always be performed to establish which radio channels can be used for the deployment and to ensure that the proposed spectrum is clean.
■ The recommended radio for deployment on AGV/AMR is the FM Mobi since it is vibration-resistant and provides M12 X-coded LAN connectors.
■ The FM Mobi can be powered using either 802.11at PoE+, or through the DC IN port, providing a flexible choice of how to power up the radio.
■ High vehicle densities result in more interference and contention in the affected areas. The OMNI-3, an antenna designed for optimum performance at low signal strengths, is most suitable for this purpose.
– The OMNI-3 is the antenna of choice for both infrastructure radios and AGV/AMR radios.
– The OMNI-3 antenna has a low gain, helping minimize interference within the indoor areas where the AGV/AMR move and it has a radiation pattern that helps eliminate ‘dead spots’ if a vehicle passed underneath it.
– The OMNI-3 antenna also has a small form factor which makes it ideal for installation on vehicles where space is limited.
■ Adjustments to the system design are needed whenever an RF simulation or an active RF survey does not show an expected level or RF coverage.
■ Always locate vehicle antennas in areas where there is as little metal as possible. Metal reflects and degrades the RF signals.
■ Radios and antennas must be quickly accessible for easy maintenance.
■ If the chosen hardware placement does not work well during initial testing, try moving the antennas to a different location.
■ Customer performance requirements (for e.g., minimum and maximum bandwidth, PLC safety cycle, VLAN, QoS, Industrial protocol in use – CIP, CIP Safety, PROFINET, PROFIsafe).
■ Collect key data as part of a site survey:
– Have a detailed floor plan or blueprint of the Warehouse or Factory Plant Floor
– Location of existing backbone network infrastructure, such as fiber drops or switches
– Metallic Object obstructing or reflecting RF signal (Robots, Machinery, Shelves, Roof, etc.)
– Any other obstructions (walls, catenary structures, buildings, etc.)
– Number and type of vehicles. Speed of vehicles.
– Intra-Cell and Inter-Cell roam requirement?
– Location and availability of power to install infrastructure APs.
– Location and availability of power to install vehicle radiosAPs(s).
– Perform a spectrum scan to determine what radio channel(s) can be used for the deployment.
■ Perform an active site survey to determine if the amount of deployed wayside CURWB infrastructure RF coverage meets both the customer and application KPI requirements.
■ What is the CIP Safety Requested Packet Interval (RPI) requirement for the automation system?
■ CIP Safety: What is the timeout interval set to? (RPI x timeout = The safety requirement of the system)
■ What is the size of the network, and how many access points are required?
■ AGV/AMR density. Total number of AGVs that need to be supported?
■ What policies are in place so that RF interference is not introduced in the future as more wireless networks are added.
■ Know which setup decisions will most likely impact wireless network performance. That means looking carefully at the PLC communications, the requested packet interval (RPI), and the time out multiplier - if these values are set too short, communication faults can be expected. Getting the RPI right is important - too fast and it’ll create unnecessary traffic on the network, overloading the network will result in triggering communication faults.
■ Determine the total network load or how many packets per second will be produced, because if the network is overwhelmed, communication faults will be triggered and the AGV(s) will stop. It’s best to have a dedicated channel that’s not used in any other wireless systems in the facility. That allows radios to talk without interference from other wireless networks.
■ Need to determine Channel Width depending on AGV density per Infra AP on the Plant Floor/Warehouse
■ Need to tune the CIP Safety Request Packet Interval (RPI) rate between Wired Safety PLC and Wireless Safety I/O to avoid connection timeouts
■ Need to tune the Timeout Multiplier between Wired Safety PLC and Wireless Safety I/O to avoid connection timeouts in case of consecutive packet loss
■ Total (Upstream + Downstream) traffic per AGV will need to be factored into the above planning
■ Any other non-AGV traffic over the CURWB Wireless network will also need to be factored into the calculations. It is highly recommended to not have any additional traffic running over the CURWB network designed to support AGV/AMR application.
PROFINET (PN) communication can also be realized over a standard IEEE 802.11 wireless connection. While some PROFINET IO (PNIO) devices have built-in wireless client capabilities, the majority of PNIOs only support Ethernet interfaces. In those cases, system integrators will need to connect the PNIO to a wireless client device that acts as a wireless adapter to communicate with the PN controller.
In order to design and deploy the right wireless solutions to support PROFINET communication, several key aspects of wireless networking need to be taken into consideration. These include L2 transparency limitations, higher latency, and radio frequency (RF) management to configure the wireless environment for optimal performance.
PROFINET real-time (RT) communication requires the underlying network infrastructure to be Layer 2 transparent in order to forward data frames correctly. The deployed wireless solution must support L2 forwarding.
Some other variables to consider:
■ The number of PNIO devices to integrate
■ The scale of the wireless network (the number of wireless devices to deploy)
■ Device mobility requirements
Figure 29 CURWB L2 Fluidity Architecture for Factory AGV/AMR
The Purdue Model for Control Hierarchy (reference ISBN 1--265-6) is a common and well-understood model in the manufacturing industry that segments devices and equipment into hierarchical functions. It has been incorporated into many other models and standards in the industry. Based on this segmentation of the plant technology, the International Society of Automation ISA-99 2-2 Committee for Manufacturing and Control Systems Security has identified the levels and logical framework. Each zone and the related levels are described in detail below.
The Cell Area zone is a functional zone where the industrial automation devices interact with each other. It can be visualized as a kitchen area where the chefs prepare the food. The network is a critical factor because all the automation devices must communicate to ensure that goods are produced. A plant factory may have one or multiple cell zones. Each cell can have the same or different topologies. The diagram represents the one used in this design for validating the PROFINET 2.0 solution.
The Manufacturing zone comprises the Cell/Area zones (Levels 0 to 2) and site-level (Level 3) activities. The Manufacturing zone is important because all the IACS applications, devices, and controllers critical to monitoring and controlling the plant floor IACS operations are in this zone. To preserve smooth plant operations and functioning of the IACS applications and IACS network, this zone requires clear logical segmentation and protection from Levels 4 and 5 of the plant/enterprise operations.
For deployments that do not need support for inter-cell AGV roaming the recommended deployment topology is L2 Fluidity. The figure above depicts a typical L2 Fluidity mobility architecture for a single cell factory environment. A pre-requisite for L2 Fluidity is that all the CURWB devices (mesh-end gateways, access radios, and mobile radios) need to be within the same VLAN/IP subnet/L2 broadcast domain and configured with the same passphrase.
Note: L2 Fluidity can also be implemented without the use of the ring topology as depicted above. In that case the CURWB access radios can be connected to the access layer switches within the deployment.
The deployment consists of a redundant pair of mesh-end gateways. The mesh-ends can either be CURWB radios for smaller deployments of FM gateways for larger deployments. The role of the mesh-ends is to terminate the MPLS tunnels from each of the vehicles and access radios and act as a demarcation point between the wired and the wireless domains. The mesh-ends are responsible for de-encapsulating the MPLS header and then forwarding the traffic to the distribution/core switch. For the traffic originating from the wired network and going towards the mobility domain, the mesh-ends act as the “default gateway” and are also responsible for the MPLS encapsulation and forwarding the traffic to the appropriate vehicle radio.
The access radios are configured as mesh points in the L2 Fluidity mode with the same passphrase that is configured on the mesh-ends. The role of the access radios is to provide RF coverage for the mobility domain. The access radios are distributed across the area where wireless coverage is required while the AGV/AMR roam.
The CURWB radio on the vehicles is configured in “Vehicle” mode.
The CURWB network architecture is based on Prodigy 2.0, MPLS-based technology, which is used to deliver IP-encapsulated data. MPLS provides an end-to-end packet delivery service operating between levels 2 and 3 of the OSI network stack. It relies on label identifiers, rather than the network destination address as in traditional IP routing, to determine the sequence of nodes to be traversed to reach the end of the path.
An MPLS-enabled device is also called a Label Switched Router (LSR). A sequence of LSR nodes configured to deliver packets from the ingress to the egress using label switching is denoted as a Label Switched Path (LSP), or “tunnel”. LSRs situated on the border of an MPLS-enabled network and / or other traditional IP-based devices are also called a Label Edge Router (LER). The ingress node of the path classifies incoming packet according to a set of Forwarding Equivalence Classes (FEC); when a packet matches a class, it is marked with the label associated with the particular class and then forwarded to the next node of the sequence, according to the information configured into the Forwarding Information Base (FIB) table of the node. Subsequently, each intermediate node manipulates the MPLS label(s) stored into the packet and then it forwards the data to the next node. The egress node finally removes the label and handles the packet using normal IP routing functions.
The FIBs on the different nodes of the network are managed by means of a Label Distribution Protocol (LDP) that is the primary component of the so-called network control plane. Fluidity relies on a custom label distribution protocol that provides automated installation of LSPs among the different nodes of the network; this ensures that each node can be reached from any other node.
In traditional MPLS networks, whenever the network topology changes for any reason, the FIBs of the nodes involved must be reconfigured to adapt to the new paths. This is usually performed using the standard label distribution protocol signaling available.
In a mobility network scenario, the handoff process can be assimilated to a network topology change, where a link is broken and a new one is created like in Wi-Fi. The standard mechanisms to detect the change and reconfigure the nodes are, however, too slow and data-intensive to provide adequate performance in a real-time constrained scenario such as high-speed mobility. In particular, the whole reconfiguration latency and the number of messages exchanged should be minimized to reduce the chances that some data packets are lost in the process.
To mitigate the mentioned issues, Fluidity implements a fast handoff solution that is able to provide very fast path reconfiguration with latency in the order of one millisecond. The considered mechanism is proposed as an extension to the existing control plane of the network and it is based on a specific manipulation technique concerning the MPLS FIB tables of the nodes.
The scheme proposed allows mobile nodes, and client devices attached to them, to maintain their IP address throughout the mobility process. Besides, all nodes are part of a single layer-2 mesh network. The layer-3 handoff process is seamless in the sense that, thanks to a make-before-break strategy, the availability of at least one valid LSP is ensured during the handoff transitory as the network is reconfigured.
LSPs connecting to the static backbone are installed and updated whenever the vehicle performs the handoff procedure using dedicated signaling. LSPs are always present as long as a mobile radio is communicating with a fixed infrastructure radio. Labels change as it roams, but the LSPs are always present.
Figure 30 CURWB L3 Fluidity Architecture for Factory AGV/AMR
Within a Fluidity L3 design, the CURWB devices belong to different L2 broadcast domains. These domains consist of multiple subnets and are organized as separate L3 routing domains. This design needs to be implemented when the AGV/AMR needs to cross subnet boundaries from one cell/area zone to another cell/area zone, basically where there is a need to support AGV/AMR inter-cell roaming.
In this design a pair of FM gateways need to be installed in the distribution or core layer to act as Global gateways. Each cell/area zone needs to have a pair of FM gateways or CURWB radios configured as Mesh Ends. The traffic between the Mesh Ends in each cell and the Global Gateway is encapsulated within an L2TP tunnel. The CURWB radios are configured to belong to the subnet where they belong.
L3 Fluidity enables scalability across multiple Cell Area zones each mapped to their own subnet. Fluidity L3 sits on top of the existing networks and ‘flattens’ subnets via L2TP encapsulation between the Global Gateways and the Mesh End(s) within each Cell Area zone, allowing seamless routing and end-to-end connectivity for an AGV/AMR moving between different cell area zones and the core network.
Communication between the wayside infrastructure subnets and the Global Gateway network is provided in the routed IP network. Communication between the Global Gateway network and vehicle networks goes through the MPLS overlay and L2TP tunnels. Each radio network on the wayside corresponding to an individual Cell Area zone needs at least one Mesh End. Dual Mesh Ends can be deployed for redundancy if needed. Each of these Cell Area zones belong to different broadcast domains.
CURWB radio on-board the vehicle should be configured with a static route of each local subnet on the vehicle. This allows the vehicle radio to advertise these addresses back to the network core (Global Gateways) for network convergence. The IP address of the on-board router should be used to configure the default gateway for the vehicle CURWB radio.
Each Global Gateway should be configured to connect to the L2TP WAN address of each Mesh End. L2TP encapsulates the original IP/MPLS payload.
Both Global Gateways should be configured with the FM-TITAN plug-in enabled, enabling ta Virtual IP to be configured. This enables the network to stay converged in the event of a hardware failure on the primary helping the secondary global gateway to seamlessly take over. Additionally, the L2TP address of each Mesh End should be entered in list format on each Global Gateway to build the L2TP tunnel from the Global Gateway to each Mesh End.
The Mesh End within each Cell/Area zone can also be deployed in a pair to provide high-availability if desired. In that case both the Mesh Ends will need to be configured with the FM-TITAN plug-in enabled. The L2TP address of each Global Gateway should be entered in list format on each Mesh End to form the L2TP tunnel to the Global Gateway.
Note: L3 Fluidity deployment will not work for PROFINET requires L2 adjacency. PROFINET deployments must always use leverage the L2 Fluidity architecture design described above.
TITAN is the CURWB proprietary fast-failover function, allowing data exchange to resume almost instantaneously in case of a transceiver, switch-port and/or CURWB device failure. This unique feature can virtually guarantee uninterrupted service in mission-critical applications where safety and/or operations would otherwise be compromised by failure of one or more non-redundant radio transceiver units.
Leveraging CURWB MPLS-based protocol, TITAN is able to achieve device failovers within 500 mSec on both Layer 2 and Layer 3 networks.
Once enabled, TITAN is completely autonomous and ensures stable and reliable connectivity without the need for human supervision. If a primary device fails, the designated secondary device automatically takes over the exchange of data. If the failed primary device comes back online, the secondary device automatically reverts back to its standby role.
For faster convergence and High Availability using hardware redundancy, it is highly recommended to install theTITAN plug-in on all the gateways and radios in the deployment.
TITAN can be enabled on:
■ Vehicles with Dual Radios in a Dynamic Multi-Frequency deployment
■ On Way-side Infrastructure radios to speed-up the Mesh-End check process in case of failure of the radio wired interface or wired network failure preventing from that radio being able to reach the Mesh End node
■ On FM- Gateways that are used either in Mesh End or Global Gateway role
Enable GARP when enabling TITAN plug-in to advertise the secondary’s MAC address on failure of the primary unit.
Figure 31 Redundancy and High-Availability at the Core Layer
Cisco StackWise Virtual is a two-node solution providing a Unified Control Plane Architecture by assigning one switch as active and the other as a hot-standby. Both the switches play an active role when it comes to data forwarding. Two Cat- switches are connected together using a StackWise Virtual Link (SVL). The SVL helps bring the two switches together forming a single logical switch. Both the switches can be managed as a single entity. Since the control plane, management plane, and data plane are integrated, the system behaves as a single switch. The advantage of configuring the switches in a StackWise pair is that it provides hardware redundancy and fast failover.
It is highly recommended to purchase and apply the TITAN high-availability plug-ins for a pair of redundant FM- Mesh Ends to be used within the AGV/AMR deployment.
Once configured, TITAN is completely autonomous and ensures stable and reliable connectivity without the need for any human intervention. If data exchange ceases because of the failure of the primary mesh-end device, TITAN will detect the failure and re-route the traffic through the designated secondary device, re-establishing connectivity within a maximum of 500 mSec. When the failed primary mesh end device comes back online, the secondary mesh end device automatically reverts to its standby role.
It is highly recommended to power each of the FM gateways using a different power source and connect them to different switches within the StackWise pair. This provides protection against power outages and switch hardware failure.
All CURWB units connected to the same wired broadcast domain and configured with the same passphrase perform a distributed primary election process every few seconds. The primary unit constitutes an edge point of the Fluidmesh MPLS network, i.e. a device where the user traffic may enter or leave the mesh. Secondary units act as MPLS relay points.
For each neighbor, the algorithm computes a precedence value based on the role of unit (mesh-end or mesh-point) and its mesh-ID. Mesh-ends are assigned a higher priority than mesh-points and, among the same priority, the unit having the lowest mesh-ID is preferred. The election mechanism relies on a dedicated signaling protocol which constantly runs in the network and it guarantees that all units elect the same primary.
During normal operation, the primary and secondary mesh-ends constantly communicate to inform each other about their status and to exchange network reachability information. In particular, the primary periodically sends updates to the secondary regarding its internal forwarding table and multicast routes.
When the primary mesh-end unit fails for any reason, a timeout expires on the secondary after not receiving keepalives for a configurable interval (typically between 50 - 200 mSec). At that point, the secondary becomes the new active mesh-end taking over the role of primary and it executes the following actions:
1. Issues a Primary Change command to inform all other units on the same wired network that the primary has changed. The message is propagated to mobile units as well using an efficient distribution protocol.
2. Updates the internal MAC and MPLS forwarding tables. This step is performed using a patented fast-rerouting technique that provides seamless performance.
3. Sends gratuitous ARPs for the on-board devices on its ethernet/fiber port. This forces the network switch to update its MAC forwarding table (CAM) so that it will send traffic for on-board destinations through the port connected to the new primary.
When the other units receive the Primary Change command from the secondary, they perform the internal seamless fast-rerouting procedure (step 2), so that the traffic can be immediately forwarded with no additional delay and signaling required. This allows for fast network reconvergence, with an effective end-to-end service disruption below 500 mSec.
When the primary mesh-end is recovered, it initially scans the network for the presence of an active secondary mesh-end. If the detection is positive, then the unit enters an inhibition mode where-in the secondary remains the current edge point of the infrastructure for a certain amount of time (default 70 seconds). During this grace period, the primary receives updates from the currently active secondary mesh-end and acquires full knowledge about the state of the network. After that, it switches to being the primary node following the same procedure described above for failover.
Just like the Mesh-end gateways, the Global gateways within a L3 Fluidity design are deployed as a redundant pair. Each global gateway should be configured with the TITAN high-availability plug-in to enable fast convergence in case of a hardware or network link failure.
With the system in normal condition (all devices up and running), this is the expected scenario between the global gateways and each L3 Fluidity trackside cluster:
■ L2TP Tunnel between Primary Global Gateway and Primary Mesh End – CONN
■ L2TP Tunnel between Primary Global Gateway and Secondary Mesh End – IDLE
■ L2TP Tunnel between Secondary Global Gateway and Primary Mesh End – IDLE
■ L2TP Tunnel between Secondary Global Gateway and Secondary Mesh End – IDLE
Global Gateway Failure: In case of failure of Primary Global Gateway, the L2TP tunnels between itself and the primary Mesh End of each cluster will become IDLE, while the L2TP tunnels between the secondary Global Gateway (elected the new Primary) and the primary Mesh End of each cluster will become CONN.
Mesh End Failure: In case of failure of Primary Mesh End in a cluster, the L2TP tunnels between itself and the primary Global Gateway will become IDLE, while the L2TP tunnels between the secondary Mesh End (elected the new Primary) and the primary Global Gateway will become CONN.
Leveraging the Link Backhaul Check feature, an access radio unit detects a carrier loss on its Ethernet/Fiber port hence losing its ability to deliver mobility traffic to the mesh-end. The affected radio unit immediately advertises its status as ‘Unavailable’, by transmitting a ‘handoff inhibition’ message over the wireless channel. Upon receiving the ‘handoff inhibition’ message any existing mobile radios connected to this particular radio unit will try and search for another access radio to connect to. All mobile radio units currently connected to this unavailable access radio will find and connect to an alternative access radio unit within a few hundred milliseconds, typically within < 400 mSec. Also any handoff attempts from any other mobile radios to this affected access radio will be rejected. It is highly recommended to enable the Link Backhaul Check feature on the access radios within an AGV/AMR deployment.
Figure 32 Link Backhaul Check – Handoff Inhibition
In the figure above it is shown that the link between the infrastructure radio and the directly connected switch is down. Assuming that the radio is not powered using PoE but via an external power source the radio is still up and providing good wireless connectivity to the vehicles. However since the wired link is down and the radio is not able to forward traffic to the wired network, the radio goes into handoff inhibition mode and will reject any handoff attempts from vehicles.
Leveraging the Mesh-End Backhaul Check feature, an access radio unit detects that it is not able to reach the active mesh end. This failure is triggered when L2 MAC reachability is lot to the active mesh end for 250 mSec. The affected radio unit immediately advertises its status as ‘Unavailable’, by transmitting a ‘handoff inhibition’ message over the wireless channel. Upon receiving the ‘handoff inhibition’ message any existing mobile radios connected to this particular radio unit will try and search for another access radio to connect to. All mobile radio units currently connected to this unavailable access radio will find and connect to an alternative access radio unit within a few hundred milliseconds, typically within < 400 mSec. Also any handoff attempts from any other mobile radios to this affected access radio will be rejected. It is highly recommended to enable the Mesh-End Backhaul Check feature on the access radios.
Figure 33 Mesh-End Backhaul Check – Handoff Inhibition
In the figure above it is shown that the directly connected switch of a CURWB access loses its wired connectivity to the core switch, the infrastructure radio is powered on and providing good coverage and connectivity to vehicles but since the radio is not able to forward traffic to the mesh end located within the control room, it will go into handoff inhibition mode and reject any handoff attempts from vehicles.
TITAN high-availability is not just applicable for mesh ends. In scenarios where two CURWB radios are deployed on-board a vehicle, the two radios can paired-up together by applying the TITAN plug-in to provide hardware redundancy.
The on-board failover process is very similar to the mesh-end one and it encompasses the same steps described in the previous section by just swapping the infrastructure and the on-board networks.
The main difference is that when a mobile unit becomes the new primary after a failure or recovery event, it executes the following additional actions:
1. If the automatic vehicle ID feature is enabled, it computes a new Vehicle ID and forces the update on all the on-board units accordingly.
2. It performs a forced handoff procedure instead of sending a Primary Switch command to update the infrastructure network even more efficiently.
If you want to learn more, please visit our website Latent AGV.