Connected out in the fields
Highly automated, interconnected and compact agricultural machines that are flexibly deployed in the field in a swarm: what sounds like the distant future could become reality in just a few years. In the Feldschwarm (field swarm) research project, IAV and ten other partners from industry and science have joined forces to develop just such ideas and bring them to the field.
Robert heads the Feldschwarm research project for IAV, which focuses on highly automated, compact and flexible machine swarms instead of a few large and specialized agricultural machines. Key advantages: Lower vehicle weight protects the soil, the failure of a vehicle can be compensated for more easily, and interchangeable working tools increase machine flexibility. At the same time, the swarms are operational around the clock, and the classic role of the driver is transformed into a system supervisor and manager responsible for the entire network.
«However, thanks to the high level of automation, the machines also contribute towards more resource-efficient and ecological practices in agriculture.»
— Expert in perception and simulation at IAV
“Farmers can use the swarms in a more targeted way – for example, when using sprays or fertilizers,” says Robert. The prerequisite for targeted use is that the agricultural machines not only separate the wheat from the chaff, but also actually recognize and distinguish whether the plants are healthy and the fruit is ripe, or whether objects or people are blocking the way, so that they can draw the right conclusions accordingly.
In the Feldschwarm project, IAV develops and is responsible for intelligent environment perception – the sense of sight for agricultural machinery. The agricultural field poses particular challenges for the engineers. “The classic methods for environment detection and collision avoidance from the automotive sector, such as simply recognizing objects, are not enough,” says Robert. “Environment perception must work so precisely that it can not only recognize but accurately classify a wide variety of objects, objects and plants.” To do this, Robert and his team first categorized objects into three groups: dynamic objects such as vehicles, people and animals; static ones such as utility poles or drainage ditches; and semi-static ones such as straw bales or briefly parked work equipment. Static and semi-static objects can be mapped permanently or briefly in the second step. “The machine can remember the position of the object and move around it without intervention,” Robert says. “It gets difficult with dynamic objects, because they cannot be mapped and can only be reliably classified through intensive training of the environment recognition.”
Photorealistic simulation with varying objects, scenes, weather conditions.
Automatic labeling of objects and generation of training data for neural networks.
To do this, the IAV experts trained neural networks and fed them image material over and over again to increase the hit rate and the correct assignment of objects by the agricultural machine’s sensors – lidars, radars and cameras. Robert and his team had to break new ground to do this, because in the agricultural sector, unlike for road traffic, there is hardly any freely available training data to date. “In addition to real data sets, we created numerous photorealistic simulation environments, with varying objects, scenes and weather conditions, to automatically generate training data for the networks. This allows us to teach the system what a corn plant is, for example, or what weeds look like, or which objects are trees. With this comprehensive set of virtual and real training data, we have prepared the neural networks for the first field trials in the real environment,” Robert explains.
Those trials are now on the horizon. The test vehicle was set up during the winter months and expanded to include the components and sensors for environment perception. The agricultural technology of the future is moving one step closer.
Feldschwarm® – autonomous field modules for resource-saving resource-conserving agriculture
By July 2020, the seven companies and four research institutes in the Feldschwarm consortium have developed the basic technologies for autonomously operating attachments in agricultural technology. This technological shift in agriculture toward lightweight, flexible, highly automated, electrified equipment systems offers opportunities for central Germany’s agricultural machinery sector to re-establish itself on the global market with high annual sales.
As a joint partner, IAV is responsible for the development of environmental sensor technology and sensor data fusion, taking into account the specific operating conditions of agricultural vehicles. In addition, adapted data maps as well as navigation components for route planning of the entire swarm are to be developed.
Project partners:
Reichhardt Elektronik GmbH, Indikar Individual Karosseriebau GmbH, EIDAM Landtechnik GmbH, BITSz electronics GmbH, ILEAG e.V., Institute for Light Electric Drives and Generators, John Deere ETIC, Fraunhofer Institute for Transportation and Infrastructure Systems, Fraunhofer Institute for Machine Tools and Forming Technology, IAV GmbH, Raussendorf GmbH, TU Dresden
The article was published in automotion 01/2021, the automotive engineering magazine of IAV. Here you can order the automotion free of charge.
