Issue 4
Track work inside the rail corridor is inherently high risk. Current practice often requires personnel to enter the danger zone to install safety and isolation equipment before works commence. This, coupled with the projected long-term increase in network utilisation, creates a scenario of foreseeable increased worker risk that should be addressed.
This project, which was jointly funded by ACRI/NTRO and iMOVE and undertaken by the University of Newcastle, sought to develop enabling smart technologies that minimise human exposure to risk within the rail environment. It involved development of a general-purpose sensing platform that can accurately determine its position within the rail corridor, detect changes and track dynamic objects, including people, as they operate within the space.
The challenge was to develop a sensor that had the following capabilities:
Furthermore, it needed to be a lightweight, standalone solution that performed these functions in real time, with the capacity to be applied to a variety of platforms for fixed location and dynamic applications.
This paper presents the efforts on developing a solution to this problem, conducted across two projects. In the first project, a prototype of the robotic trackside device was constructed. This served as the platform for the development of the real-time perception sensing capabilities undertaken in the second project.
A prototype robotic trackside device was developed as part of the first project to investigate what can be done in real-time with lightweight hardware and small computational resources. Figure 1 shows the trackside device that was developed.
Fitted with GPS, 4 Intel RealSense cameras, an advance spatial Inertial Navigation System (INS), a small onboard computer and wheel treads, this trackside device can move about in the rail space and climb over ballast onto rail tracks. It also has internal track wheels that can be lowered to deploy the device to the rail for rail-based applications. This capability gives it the potential to undertake remote rail inspections or install safety equipment with the addition of appropriate peripherals. Additionally, it was used to build an internal map representation of the rail environment by performing simultaneous localisation and mapping (SLAM).
Continuing from the first project, the sensing capability was further improved in the second project. The developed sensing platform had three primary functionalities:
Figure 2 shows the sensing unit, which includes Advanced Navigation Spatial Dual GNSS/INS for global localisation, a Baraja Spectrum solid-state LiDAR for local localisation and a lucid global shutter camera for classification. While it could have been mounted on the trackside device developed in the first project, this advanced sensing unit was developed for potential use in a range of static and dynamic applications. As such, the unit was built into a trolley that could be moved by a person to map the local precinct.
Localisation is the process of determining where on Earth the sensing platform is, similar to the GPS function of a smart phone. In many rail applications, however, GPS signal is either degraded by reflections (eg. nearby buildings and mountains) or completely unavailable (e.g. in tunnels). For this reason, the project sought to develop a solution that could operate without ongoing reliance on GPS signals.
To complement GPS localisation, the team utilised existing network scan data obtained from network operators. A two-stage localisation solution was developed using this additional information. First, GPS was used to obtain an initial estimate of the sensor’s location in space. Once this estimate was obtained, the LiDAR was used to localise the device with respect to the existing network scan data. Once a location lock was received, the device could operate in previously scanned environments without the need for GPS signal. However, when the device lost track of its location, the localisation process restarted and the user was informed.
To enable real-time application, the file size of the network scan data could not be too large and, as such, the project team utilised two compression methods, one for localisation and another for change detection (Ferguson et al. 2020).
When a robotic system revisits a previously mapped environment, it is likely that spatial changes will have occurred. These could be the result of new installations, vegetation growth or the movement of dynamic objects such as people or rolling stock. The sensing platform was developed to detect these changes by comparing the LiDAR measurements with the existing network scan information. Figure 3 shows an example of such change detection where white squares are used to indicate detected changes in an otherwise known environment, indicated in green. In this case, the white squares are the result of a person walking nearby.
While the change detection was able to effectively detect that spatial change had occurred, it did not provide direct insight as to what the nature of the changes were. To obtain this additional insight, a camera was used in conjunction with the LiDAR to classify detected changes into several categories. Detected changes were clustered and the location of the changes on the camera image was specified. The corresponding image of the changes was then passed to a Neural Network (AI) for classification (e.g. person, vegetation, car or train).
If the detected change was dynamic in nature (e.g. person), a tracking algorithm that predicts future movement based on past motion was implemented. This algorithm kept track of the movement of known objects and, if they moved out of view, provided an estimate on the rough location of the object within a world map. Figure 4 shows a visual overlay from the LUCID camera with tracked objects classified as people in the top frame. The bottom frame shows the tracking output that overlays the location of detected people on a map of the surrounding environment.
Based on consultation with industry stakeholders, several applications for both a fixed-location sensing platform and dynamic sensing platform were identified. Some of the key applications identified were as follows.
A statically mounted sensing platform can be utilised in the context of trespass and unauthorised access detection in the rail corridor. LiDAR sensors can accurately measure distances and create a 3D map of the rail corridor, allowing for physical intrusion detection. The camera component can provide visual surveillance, capturing images or videos of individuals attempting to enter the restricted area. To deter offenders, the system can trigger an alarm or siren and initiate recording as soon as unauthorised personnel enter the rail corridor. This immediate response can help deter potential intruders and promptly alert security personnel to unauthorised access.
By combining LiDAR and camera technology, the system offers an effective solution for detecting trespassing and unauthorised access in the rail corridor. It provides real-time monitoring, accurate distance measurement and visual evidence, which enhances security measures and the safety of the rail infrastructure. Similar application can be used for level crossing monitoring to ensure that a crossing is clear of pedestrians or vehicles as a train approaches.
The sensing platform can be used in either a static or dynamic implementation for worksite monitoring. The device can identify the location and movements of workers within a job site, which enables several real-time monitoring possibilities:
In each case, an audible alarm can be triggered, alerting workers to the breach so that corrective action can be immediately taken.
Both the dynamic and static systems can record images of environmental changes and send them to the operator, providing crucial data for hazard detection after a weather event. By capturing real-time images and accurately measuring distances, the systems can track and document significant changes in the environment caused by the weather. These images can be sent to the operator, enabling them to assess the extent of damage, identify potential hazards and prioritise response efforts. The recorded data from the camera and LiDAR system serves as a valuable resource for efficient hazard detection and response, contributing to the overall safety and recovery process in affected areas.
The dynamic sensor system can be leveraged for vegetation monitoring along the rail corridor. This monitoring is crucial for maximising track usage and minimising maintenance work. By exploiting the data provided by this system, rail operators can optimise maintenance scheduling by conducting vegetation maintenance only when needed. Rather than following fixed maintenance frequencies, the system enables operators to identify areas with excessive vegetation growth and prioritise maintenance activities accordingly. This targeted approach ensures that maintenance work is performed when necessary, minimising disruptions to track usage and reducing unnecessary maintenance efforts. By effectively managing vegetation growth, rail operators can maximise track availability, improve operational efficiency and minimise the overall maintenance workload.
The dynamic sensor system can be effectively employed to inspect worksites after shutdown, ensuring that nothing is left behind and the track is clear for use. This sensor system offers a significant improvement over the current practice of manually walking the railway line, which can be time-consuming when covering several kilometres of track. By attaching the sensor system to a rail vehicle, the inspection process can be completed quickly, allowing for the prompt utilisation of the track. This not only enhances operational efficiency but also improves safety by minimising the risk of any debris or obstructions remaining on the track after shutdown.
The final stage of this project was to plan a pilot study. This focused on implementing two sensing units with the developed technology solutions for mapping and localisation, change detection and object tracking. The planning for this pilot included costing estimates for the platform procurement; proposed applications for initial investigation; and costing of the overall study.
The first sensing unit is proposed to be a static unit mounted to a fixed location, such as a station platform, to detect environmental changes and detect and track pedestrian intrusions. The second is proposed to be a dynamic unit mounted to a moving object, such as a vehicle that runs on the rail tracks and detects changes and intrusions in the environment, as well as detecting and tracking objects in the environment. In both cases, the system requires redesign into an enclosed unit to provide weather protection and standard mounting options to enable deployment in rail environments.
As both fixed location and dynamic applications are being considered, the sensing needs of each application will vary. Specifically, fixed location application does not require an INS as the location can be calibrated at installation and is not expected to significantly change over the lifetime of the device.
The static system would attach to a pole or a building next to a location of interest on the rail corridor. While a static system is excellent for monitoring pedestrian movements at a rail platform or level crossing, there may be issues with occlusion when pedestrians move behind signs or other pedestrians. Multiple static systems could be used so that areas surrounding a rail platform have overlapping scan areas such that regions of interest can be monitored effectively. A power source and a network connection would be needed to send pedestrian intrusion alerts to rail staff.
The dynamic system would mount to a track vehicle. This system would require a GNSS and INS unit to aid in the localisation of the unit such that changes can be detected from a known map. A power source and a network connection would be needed to send pedestrian intrusion and foliage or other significant change alerts rail staff.
While the Baraja Spectrum LiDAR was used successfully in this project to detect and track objects in real-time, this LiDAR unit may not be available for future procurement. Thus, in planning for a pilot study, several alternative solid-state or hybrid solid-state and mechanical LiDAR units were considered. While varying in their resolution, point density and price, all alternative units considered could be substituted for the Baraja Spectrum LiDAR without significant impact on device performance.
Current methods of detecting changes have looked at points that do not adhere to a known map. While this approach has been useful, there is a bank of research that has investigated using deep learning methods to classify objects in the environment directly from the point cloud. Using these approaches, objects such as people, bikes, cars and trucks can be detected and classified without image data and without the need for a known map. A future area of research could be to exploit these algorithms to provide another way of detecting people and objects that could be used in conjunction with current methods of detecting changes.
NTRO and the University of Newcastle are seeking funding to progress this real-time perception work through the implementation of the proposed pilot study. If interested in funding the pilot study, please contact Paul Bennett at NTRO, paul.bennett@ntro.org.au.
Acknowledgements
The authors would like to acknowledge Dr Tim Farnworth for his contribution to this project while at the University of Newcastle, as well as the support for the project from iMOVE and ACRI/NTRO working group participants including Arc Infrastructure, ARTC, DPTI SA, FMGL, KiwiRail, NZTA, Queensland Rail and TMR Qld.
