SRDC Research Project Final Report
Title of the Project: Automating harvester and haul-out forward progression during harvest utilizing DGPS
Project Reference Number: GRF001
Name(s) of the Research Organisation(s): Granshaw Farming, JCU, BMSLaserSat Principal Investigator’s name(s), contact phone number, address and Email address:
Mr Bryan Granshaw M/S 8268
54 Dalbeg Ayr Qld 4807 Ph: 07 47840155
The project participant/s wish to acknowledge receipt of project funding from the Australian Government and the Australian Sugarcane Industry as provided by the Sugar Research and Development Corporation.
The Research Organisation is not a partner, joint venturer, employee or agent of SRDC and has no authority to legally bind SRDC, in any publication of substantive details or results of this Project.
Executive Summary:
(Maximum 800 words. This should provide a non-technical overview of the project which could be used to communicate project outcomes in media such as the SRDC website. It should cover the following:
• Issue:
The issue that this project sought to address was to increase the performance of the harvester and haul-out during forward progression. This issue has ramifications through the entire sugar value chain as losses at this point are unrecoverable and the financial losses are spread across the industry and community. It is increasingly difficult to secure experienced
operators with the financial incentive to move to the resources sector, therefore we in the sugar industry face a skill shortage. This project investigated the use of cutting edge technology to automate critical operations to reduce the risk of operator error, to minimise fatigue due to long hours and to maximise the efficiency of the entire harvesting hauling operation whilst in the paddock.
• R&D Methodology:
Granshaw Farming, BMS Laser Sat and JCU jointly conducted this project utilising their particular knowledge in their fields along with Granshaw farming’s harvester and haul-outs.
Granshaw farming had previously invested in DGPS autosteer technology on their harvester and this was also installed in their houl-outs. The project investigated the feasibility of transmitting the DGPS data between vehicles and using this data to coordinate the longitudinal positioning of the vehicles relative to each other during harvesting and thus reduce operator fatigue.
• Project Deliverables:
This project took advantage of under-utilised technology and acquired knowledge in techniques for automating the harvester and haul-out coordination during forward
progression. The automation of the harvester haul-out coordination will result in improved harvesting practises achieving a lower cost higher value cane sample for milling and
therefore increasing sugar revenue across the value chain. The project demonstrated that it is feasible to acquire and transmit DGPS data between harvester & haul-out during forward progression, process the data and display the vehicle relative positions for the operators.
This should result in less damage to the paddock, less cane left in the field and lower operator fatigue.
• The impact of the project findings on the sugar industry and the Australian community:
This project has the potential to have a considerable economic impact as being able to accurately position the haul-out relative to the harvester during in-field harvesting will allow operators to achieve maximum billet recovery from the harvester. This in turn will improve total yield through optimally filled bins, and ultimately lead to lower fuel costs and reduced cane losses during transportation from the paddock. In addition to the economic advantages there are other factors driving harvest automation in the sugar industry. Technological aids to coordinate the operation of the haul-out and harvester will become important in future as it becomes difficult to attract and retain skilled harvester/ haul-out operators and will assist in reducing operator fatigue. Relative position indicators or assisted position control will reduce training times and skill levels needed to operate this machinery whilst optimising crop return. This type of technology could be used to enable safe operation in low visibility
situations such as at dawn/dusk and night, or when harvesting dense crops such as those obtained in dual row planting. Finally, accurate positioning of the haul-out becomes critical if the speed of the harvester is to be automatically adjusted. This will be of particular interest if new strategies in controlling harvester speed to maintain optimum yield conditions are to become practical. In such a case the harvester speed would be continually varied, making it essential to assist the movement of the haul-out to track these changes.
Background:
Precision agriculture (PA) is a broad term used to describe efficient farming practices which take into account the in-field variability of farming operations. PA relies on modern technologies like farm implement based sensors or remote sensing (satellite or aerial) to precisely measure in-field conditions including soil moisture levels and crop yield to enable optimum, fine-scale application of resources such as fertilisers, insecticides or water. Most modern PA practices will also employ accurate positioning technologies such as a global positioning system (GPS) device to accurately locate farm equipment in the field to enable the accurate delivery of these resources efficiently and effectively. Computer software systems are employed to manage the overall process including data collection, analysis and reporting.
PA has or is being adopted by a range of Australian farming sectors including the grain, livestock, wine, cotton and sugar industries. The uptake of PA has been very rapid in the grain sector where it is estimated that 30% of all broadacre crops in South Australia are now sown and/or sprayed using GPS assisted machinery. While the uptake of PA has been slower in the sugar industry, it is generally recognised that, like in many other sectors, there are substantial benefits to be achieved through the use of PA and that the use of this technology will become more widespread in the future.
The particular focus of this paper is the use of PA techniques to keep the haul-out vehicle position synchronised with the harvester during operation. Technologies are already in use to accurately position the harvester and haul-out in the paddock. So called “auto-steer” systems based on GPS navigation technology can for example be used to automatically steer equipment along a predetermined A-B line (the line or path from a start point “A” to a finish point “B”) in the field to localise soil compaction to inter-row spacing and minimise wheel damage to the ratoon crop. This requires the guidance system to have a high degree of accuracy; hence real time kinematic global positioning systems (RTK-GPS) are used for this task. RTK-GPS observes particular changes in the received satellite signals at a stationary reference site as well as the mobile unit to obtain its position to within a few centimetres of actual location. It should be noted however that current auto-steer technology used in Australia tends to operate independently of other equipment. So, even though the harvester and haul-out are both being automatically steered during harvest, each vehicle’s location is controlled independently of the other unit. In operations like sugarcane harvesting where the harvester and haul-out need to work cooperatively to cut and collect the cane billets, benefits can be gained from being able to coordinate the operation of the two vehicles. In particular, being able to accurately position the haul-out relative to the harvester during in-field harvesting will allow operators to achieve maximum billet recovery from the harvester. This in turn will improve total yield through optimally filled bins, and ultimately lead to lower fuel costs and reduced cane losses during transportation from the paddock.
Objectives:
(As stated in the original proposal, and a statement of the extent to which the project has achieved them)
The project was to investigate two areas for automating the coordination of the harvester and haul-out:
1. The control of harvester and haul-out progression during harvest using DGPS
technology. This combined with “Blue Tooth” wireless communication would enable both machines to operate along a specified “A-B” line in parallel, it would also enable the harvester to control automatically the forward position of the haul-out during harvest.
This idea was investigated and found to be feasible. A concept demonstration system was constructed which displays the relative longitudinal positions of the two vehicles for the haul- out operator. This system is used by the haul-out operator as a visual guide for haul-out
positioning, relative to the harvester, during harvesting. The system is believed to be installed in the harvester and one haul-out and to be functioning correctly although no feedback has been received from the principal investigator.
2. Control the forward progression of the harvester through the crop based on the load being sensed, therefore optimising harvester capacity to maximise throughput per hour of operation.
This was not investigated due to running out of time.
Methodology:
(Include activities and project management to deliver outputs and outcomes. Where detailed methodology has been published elsewhere, it should be summarised here and the publications should be included as Appendices to the report.
• Clearly enunciate the project process and its links to the outputs. Include where appropriate details of stakeholder participation, systems integration, implementation/adoption strategies and enhancement of human capacity.
• Specify method(s) used to evaluate success in delivery of outputs and outcomes. In particular, clearly enunciate the method(s) of evaluation used within the project to assess:
- the impact of the project,
- the learning experiences/capacity building achieved from the project, and - the implications of the project in guiding future R&D.)
Project organisation:
The project was organised with three main participants:
• Granshaw Farming Company (the principal investigator) who had the original idea for the project and provided the harvester and haul-out vehicles used during the project. As principal investigator Granshaw Farming had overall responsibility for the project including leadership, management and communications. In addition they had the expertise and experience with the operation of the harvesting equipment, including the operation of the DGPS systems.
• Brandon Muspratt Services who supplied the DGPS equipment and contributed technical expertise. Brandon Muspratt Services installed and commissioned the Trimble AgGPS Autopilot systems in the harvester and haul-outs and offered their expertise with this equipment as their contribution to the project.
• James Cook University who supplied the PhD student to work on the project, student supervision and technical expertise. The student’s role was to develop the system for coordinating the harvester and haul-out. The brief provided to the student was to investigate a method for reducing the haul-out operator’s workload by providing a positioning aid showing the relative positions of the harvester and haul-out during harvesting. The student was informed that the vehicles were all equipped with the Trimble AgGPS Autopilot systems which had a serial port that transmitted NMEA data including the vehicles location to ±2.5cm accuracy.
A summary of the system developed by the student:
System overview
To achieve vehicle co-ordination between the harvester and haul-out, the system design takes advantage of existing PA technology, namely a commercially available RTK-GPS auto-steer solution. The major components of the system can be seen in Figure 1: Relative position monitoring system Each vehicle is fitted with the auto-steer system and is connected to a Panasonic Toughbook touchscreen tablet computer that communicates with the other vehicle
using an inbuilt wireless network link in an ad-hoc configuration. The Toughbook coordinates the system operation and provides a platform to display the relative position between both vehicles.
The heart of the relative position monitoring system is the GPS. Standard GPS equipment (that is a single unaided GPS device receiving satellite signals) gives approximately 5m accuracy on a horizontal or vertical plane. An RTK-GPS system such as the one used here provides ± 2.5 cm position accuracy and is therefore well suited to precise positioning for cane harvesting.
The GPS auto-steer system used in this project is a Trimble AgGPS Autopilot RTK solution.
The auto-steer system has been used on the Cameco cane harvester employed for this study for approximately 3 years and has enabled year-on-year repeatability of harvesting along established A-B lines. By using this controlled traffic application of PA, soil compaction is localised thus maintaining soil health and ratoon crop performance. However the haul-out vehicle has only been equipped with an auto-steer unit for less than one year and so no comparison measurements are yet available for this machine, though operator comments suggest it is already a valuable tool.
Figure 1: Relative position monitoring system
The main control unit of the RTK-GPS system is the NavController II. It has been configured to send GPS standard NMEA 0183 messages via its communications port. The computer hardware is a Panasonic CF-19 Toughbook touchscreen tablet computer containing wireless connectivity.
This unit was chosen for its military standard ratings for dust, shock and vibration it will be exposed to. The built-in wireless networking using the IEEE 802.11 standard (Wi-Fi) was also suitable for inter-vehicle communications. A 9-pin RS232 serial port is used to communicate with the Trimble system.
Custom software application
To assist haul-out vehicle operators synchronise their relative position to the harvester, a visual display showing vehicle position was developed. This software application has been written in C# (C Sharp) using Microsoft Visual Studio 2008. To display relative vehicle position the software has several code modules to receive, process and visualise the necessary information.
These modules can be seen in Figure 2: Block diagram of software code. Information flow between modules is indicated by the arrows. Each module is briefly described below.
Serial port acquisition
This module receives the transmitted information from the NavController II, and temporarily stores the information internally. It passes the received information to the NMEA interpreter module.
Figure 2: Block diagram of software code NMEA interpreter
This module takes the information received from the serial port and divides it into individual messages. It checks to see if the entire message is intact and whether the checksum (an error detection method) is correct. If these are true, it then passes this information to the relevant module for further processing. At this point, only the position information (received in latitude and longitude) is passed to the vehicle location module.
Vehicle location
This module holds the immediate current position information of both the local and remote vehicles. It stores the current position of the vehicle using latitude and longitude by converting the GPS format of degrees, minutes decimal minutes to decimal degrees. If the position information is about the local vehicle, it will send it to the UDP communications module for transmission to the remote vehicle. The module then passes this information to the coordinate conversion module to convert the position to equivalent easting and northing values in metres.
The converted UTM equivalent is stored with the rest of the vehicle position information.
Coordinate conversion
This module receives latitude and longitude information from the vehicle location module, and uses mathematical algorithms to convert them into UTM Grid coordinates. This takes the GPS provided position coordinates and turns them into meaningful distances for human interpretation. It sends this information back to the vehicle location module for storage.
Determine relative position
This module takes the present location of the local and remote vehicle in the earth frame of reference and translates them to the vehicle frame of reference to calculate their position relative to each other. It provides this information to the refresh display timer module when requested.
UDP communications
This module sends the vehicle location over the wireless network using broadcast packets, and also receives location information from the remote vehicle. If it receives position information from the remote vehicle, it passes this to the vehicle location module for storage.
Refresh display timer
This module ensures that the visual display produced for the operator is updated at regular intervals. The timer initiates a request for relative position information and displays this on screen at set intervals.
Inter-vehicle communications
One of the key system requirements is for each vehicle to transmit its location to the other so a relative position display can be provided. Without this inter-vehicle communication, coordinated synchronisation using the existing RTK-GPS systems is not possible. As such, the method used to transmit information wirelessly is important to ensure it meets the needs of a real-time display including a fast response time and is engineered against information loss. The User Datagram Protocol (UDP) is a protocol used in computer networks to send and receive information between machines. It acts as a “best effort” service; that is, when the computer sends a packet of information using UDP, it expects the rest of the hardware between it and the destination to get the packet there. The difference is if a packet of information is lost in transit.
This is not a concern to UDP as it will just send or accept the next packet of information. This protocol is more suited to near real-time applications, and is analogous to watching a movie. If an image frame is lost, it just plays the next one and therefore won't be noticed – as is the case for a near real-time visual display that is regularly being updated. UDP also allows for a packet transmission method called “multicasting” – this is when the same packet of information is sent simultaneously to all computers that are listening for it. This means a harvester can communicate to any vehicles who are listening without having to establish connections to each individual vehicle. This scenario is useful when a harvester may be communicating with more than one haul-out vehicle. So selecting UDP is a sound choice as it also caters for future expansion.
Display
The display to the haul-out operator is of primary concern, as all this information processing is useless if it cannot be communicated to the operator quickly and effectively. The original screen design had a aerial view of two vehicles scaled to real-world proportions moving around, but after discussion with vehicle operators a “light bar” style aid was also incorporated to help clarify the desired haul-out position. “Light bar” is a term used in auto-steer solutions to describe a bar or row of lights that act as a gauge, showing how far off centre from the A-B line the operator is. The proposed Graphical User Interface (GUI) can be seen in Figure 3. The overall vehicle position (two vehicles) and the forward position relative to the harvester (light bar) is shown, however other information such as the current speed and distance between vehicles can also be displayed.
The light bar has three fill points marked in black. The fill point on the light bar that is closest to the top of the screen represents the front section of the bin, and the fill point closest to the bottom of the screen is the rear of the bin. The haul-out operator drives up to the harvester and stops when the lights on the light bar reach the desired fill point. At this stage, the harvester operator signals to the haul-out driver when to move to the next fill point.
In field data collection and results
In-field tests were done to check that the hardware and the software were “talking” with the commercial auto-steer units properly. These tests required small programs to be written to check the connection and log the information sent by the NavController II. The logging program that
was written for this task uses the same code as the software to read information from the NavController II. This was done to test that the software would work as anticipated.
Several test trials were then conducted in the paddock with the harvester and haul-out at different relative positions and moving at varying speeds. These trials were also video recorded.
The logged data will be input into the software and acts as a real-world simulation. The computer display can then be compared with the corresponding video footage to confirm the relative position is being updated correctly.
Inspection of the logged data when the vehicles were stationary showed the maximum difference in horizontal and vertical positions to stationary was less than 1 cm for the harvester, and 1.2 cm horizontally and 2.5 cm vertically for the haul-out.
We think that for this system to operate effectively, the haul-out will need to be within ± 15 cm of the each desired fill point. As the position accuracy of this system is ± 2.5 cm, it is expected this target will be easily met. Therefore logged data from this trial shows position information to be sufficiently accurate to present the relative distances of the vehicles to the operator.
Figure 3: Graphical User Interface display Current Status:
The system is installed in the harvester and one haul-out and is believed to be operating correctly although no feedback has been received from the principal investigator.
Problems with the project:
The development of this system should have been straight forward however a number of issues arose that resulted in major delays to the project. The delays in the project have been attributed, by some, solely to the student however I believe that there are other factors that were major contributors to these delays and these also need to be considered.
I will start by outlining the relevant issues:
• The programming environment – the system is written in C# under the .net environment.
This language and environment was chosen, by the student, to take advantage of the wireless networking capabilities of the toughbook computers and, for a proof of concept system, was a sound decision however it did result in a number of unforseen difficulties that took considerable time to overcome. The .net environment is quite demanding, particularly for real-time applications such as this, and requires a considerable level of expertise and understanding of
wireless networking and multitasking. It takes time to develop these skills particularly starting from a low threshold and the student underestimated the task.
• Auto-steer data – the Auto-steer was specified, by the manufacturer, to transmit absolute position information with an accuracy of ±2.5cm, however whilst all the units that were installed in the harvester and haul-outs appeared to be transmitting correctly but were in fact defective and were eventually shown to be transmitting standard uncorrected GPS position information with an accuracy of several meters and more importantly a slowly varying random drift. This resulted in confusing and erroneous results when the system was field tested which could not be recreated in the lab. These results caused major stress, extra work and delays for the student who understandably assumed that the units had been installed and commissioned properly and would therefore be working correctly and that the custom software would be responsible for the problem. A three tiered approach was required to track down this fault:
o Firstly the student undertook a comprehensive review of the system that he had built and determined that both the logic and methods that had been adopted were sound.
o Secondly the student constructed a sophisticated simulator to show that his custom software was working correctly. This showed that whilst the system gave correct results with artificially generated data it gave the same erroneous results (as seen during field trials) when recorded data from the field trials was used. This then led him to suspect that the data recorded in the field trials was faulty. After first reviewing his recording techniques he then proceeded to examine the data coming from the auto-steers.
o Finally the student constructed a second custom system to record & analyse data transmitted from the auto-steers in a new series of field trials. Analysis of this data proved that the auto-steers were not working correctly. The problem was eventually traced to firmware upgrades in the auto-steers that had switched off the DGPS data transmission in favour of the lower accuracy standard GPS data. I believe that some of the auto-steers are still not working correctly.
Solving this problem took many months and is one of the primary reasons for the delays in the project. Whilst the eventual solution was straight forward, solving the problem required a high level of critical analysis, patience and skill and the student should be commended for his work.
It should also be noted that this issue coexisted with the 7 meter offset issue further increasing the difficulty of the problem (see Current implementation of the auto-steer below).
It has not been explained, by the principal investigator, why it was left to the student to solve this problem and also why the equipment supplier was not asked to verify correct operation of the equipment when it was installed. This would seem to be the logical course of action as the supplier is the one most familiar with its operation & specification and was well paid for the equipment.
• Operational specification – no operational specification for the current operation of the harvester & haul-out or for the system to be developed was provided by the principal investigator. This meant that all information regarding the system had to be gathered verbally from the principal investigator usually during casual conversation. Examples of required information are:
o Vehicle dimensions o Vehicle separation
o Number and position of fill points and accuracy required o Method of moving between fill points
The lack of a specification complicated the design process and lead to delays as the design had to be constantly modified to incorporate the new information as it was made available.
• Current implementation of the auto-steer – the auto-steer as implemented on the harvester has a 7 meter offset added to the vehicle position to allow for the length of the vehicle however this information was not provided to the student when the system was being developed. When the system kept displaying a 7 meter longitudinal error the student spent a considerable period reviewing his software and conducting trials before mentioning the problem to the principal
investigator who then informed him that the offset had been included in the auto-steer initialisation. This problem lead to further delays with the project.
These issues can be attributed to two main factors, the project leadership/management and communications. The main problem with the project management was that the roles and responsibilities of the participants were not defined, for example members should have been assigned responsibility for writing an operating specification and verifying operation of the auto-steers. All communications in the project were verbal and unstructured. There should have been regular meetings held with all the participants to allow discussion of progress and problems. If these simple procedures had been followed most of the major delays experienced in the project could have been avoided.
Outputs:
(Including knowledge, skills, processes, practices, products and technology developed)
The project has demonstrated the feasibility of using DGPS technology to automate the forward progression of harvester and haul-out during harvesting. The project used existing off-the-shelf hardware integrated with custom software to produce a graphical positioning aide that can be used by the haul-out operator to show the relative positions of the harvester and haul-out. Use of the aide should result in a considerable reduction in operator fatigue.
Intellectual Property and Confidentiality:
(Detail any commercial considerations or discoveries made, and means of protection (eg patents) undertaken or planned Specify any information that is to be treated as confidential, to whom and for how long.)
N/A
Environmental and Social Impacts:
(Including any expected or actual adverse or beneficial environmental or social impacts of conducting the project and/or implementing its findings)
N/A
Expected Outcomes:
(Including assessment of the likely impact for the sugar industry in Australia and elsewhere and where possible the cost and potential benefit to the Australian sugar industry and/or the community. Qualitative and/or quantitative baseline data collected in the early stages of the project and compared with data collected towards the end of the project should be analysed and presented to demonstrate impact, learnings and additional outcomes of the project.)
This project has the potential to have a considerable economic impact as being able to accurately position the haul-out relative to the harvester during in-field harvesting will allow operators to achieve maximum billet recovery from the harvester. This in turn will improve total yield through optimally filled bins, and ultimately lead to lower fuel costs and reduced cane losses during transportation from the paddock. In addition to the economic advantages there are other factors driving harvest automation in the sugar industry. Technological aids to coordinate the operation of the haul-out and harvester will become important in future as it becomes difficult to attract and retain skilled harvester/ haul-out operators and will assist in reducing operator fatigue. Relative position indicators or assisted position control will reduce training times and skill levels needed to operate this machinery whilst optimising crop return. This type of technology could be used to enable safe operation in low visibility situations such as at dawn/dusk and night, or when harvesting dense crops such as those obtained in dual row planting.
Future Research Needs:
Recommendations:
(Including activities or other steps to further develop, disseminate or exploit the Project Outputs, and/or to achieve benefits)
The system as developed demonstrates the feasibility of using DGPS position information to coordinate the operation of vehicles in the field, however for the system to achieve its potential further development is required to firstly increase the system capabilities to controlling the haul- out forward speed rather than just displaying the relative positions and then developing a commercial product. It has been suggested that the system could be easily modified to control the haul-out position by the addition of an automotive cruise control however this is not the case and a considerable design effort would be required to achieve this.
List of Publications:
(Copies of substantive publications from the project should be included as Appendices. Where the project involves a student and the thesis is relevant to the project this should be referred to in the report and an electronic copy of the thesis sent with the report or as soon as it is available.)
Ruxton, Adam, and Grabau, P.J. (2009) Automating harvester and haul-out forward progression during harvesting using RTK-GPS. Proceedings of the Australian Society of Sugar Cane
Technologists, 31 . pp. 355-364. ISSN 07-0822.
Peter J Grabau 1st November 2010
