Mahlathini Smallholder Decision Support System

https://dss.mahlathini.org

Technologies used

Implementation details

Design considerations

User interface – to maintain a consistent styling that is approachable, easy to read and modern, Angular’s Material design has been used. This provides a formalized design and interaction system based on Google’s Component Dev Kit. Smooth animations provide a better user experience

The web application has been designed to be primarily client-side. Client-side calculation and rendering provides instantaneous feedback to the user – an important part of this system.

Input fields are grouped such that relevant fields are grouped together, or at least within close proximity. This helps the user cognitively shift subject-matter fewer times, reducing cognitive load.

Information that is difficult for the user to input, such as soil texture, soil organic carbon content, slope percentage and agro-ecological zone are automatically derived from GIS database data. This allows the user to focus on parts of the survey that are more relevant to the user (such as demographic details) and easier to fill in.

Map GIS selection section

While the system automatically finds these values, the system is designed to be flexible for the user, and allows them to change the values if they believe the data the system uses for their location is incorrect/inaccurate.

The map uses hybrid tiles, rather than purely vector road maps or raster satellite tiles. This enables the user to find locations easily by being able to see place labels and roads, while still showing natural elements. Users can find the location they are looking for through steps of increasingly fine-grained control by using the search box, looking for roads or place names, and then looking at the satellite imagery to find an exact location, respectively.

Map search

Rather than requiring exact number/value inputs (aside from a few exceptions), the input has been reduced to multiple choice where possible. This makes input easier and faster for the user, and reduces the chance of user error. It also exposes the underlying functionality of the decision support system, showing which variable boundaries affect the system, which might help inform the user. Exceptions include dependency ratio calculation, where the number of adults and children fields are numeric inputs – dependency ratio is a difficult concept for users to understand and is calculated for them.

Farmer information section

Skeleton loading screens have been implemented for both before the Angular application is bootstrapped, and before the page loads. This helps increase the perception of performance.

GIS

Reliance on external APIs has been kept to a minimum for this system. If a third party were to shut down or start charging for their previously-free services, it would require developer time before the system could become operational again.

As such, as much data as possible is stored and delivered from the server to the client.

Two external APIs are being used:

  1. OpenStreetMaps Nominatim – provides a simple API for querying locations by name, allowing the user to search for map locations.
  2. Google Maps Satellite tiles – hybrid (photo with vector overlay) map tiles for the map

Implementation of GIS

Details about ASC reader implementation can be found here

Server-side computation is very expensive, but storage and delivery of files is not. To deliver GIS information for a set of coordinates, large raster files must be traversed until the correct pixel coordinate is found, and then return the value at that pixel. These raster files can easily reach 3GB for South Africa alone for each variable (one for altitude data, one for soil texture, etc.).

Since no server-side calculations are happening, these large raster files are split up using custom software on the desktop. The files are converted into ASC format (geographical ASCII text format) which is a human- and computer-readable format. The tile is then split up into a user-defined number of smaller tiles, and an index is built to determine which tiles correspond to an area of coordinates.

When a set of coordinates is queried by the system, the index is loaded, finds the cell in which those coordinates fall, and then loads the tile that cell points to. This results in a reduced download size of multiple gigabytes to only a few hundred kilobytes, making the system significantly faster and using a fraction of the bandwidth.

In addition to partitioning of the map data, it is also compressed using gzip, giving a significant reduction in storage usage and bandwidth cost to the client.

Services implemented for reading ASC files automatically categorise z-values from the ASC files into enumerations defined in the system source. An exception is the SRTM Shuttle Radar Topology Mission data which automatically samples z-values from an area  the size of the farm selected to find the slope of multiple points, rather than assuming the entire farm is on one slope percentage.

PDF Export

PDF export was accomplished using the html2canvas and jspdf libraries. Each practice’s information to be rendered is created in an invisible element, converted to a canvas using html2canvas (special care is taken to ensure that all css is compatible with html2canvas’s limited specification). The canvas is then converted to a png, and added to a pdf object created using jspdf. Page breaks are programmatically created when images will not fit on the same page.

System Flexibility

The DSS input data is simply stored in an Excel spreadsheet. This makes it easy for maintainers to add new practices or update existing ones in an interface that is familiar to them. Practice information can be easily added and updated, and is stored in a simple JSON format defined in a JSON schema file.

Due to the use of enumerations, the values used in the survey questions are the exact values used in calculations. The values can be changed at any time and will not affect the rest of the system, making it easy to make changes.

Angular provides a modular framework, meaning more features can be added or changed at a later stage.

Implementation process

DSS was provided in Excel xlsx spreadsheet format with documentation in Word docx format. Powerpoint pptx presentations were provided containing individual practice information. Some of the GIS data was provided which would be used for production, namely AEZ data. Other data was a subset of the full datasets, and not of the entirety of South Africa.

Designs were created by hand, and multiple iterations considered.

Research was done to find GIS data and APIs. After not finding sufficiently stable or open APIs, the decision was made to host all data server-side.

Interfaces and enumerations were created for data present in the excel dss. This allows for future flexibility and easy changing of visual values without affecting underlying implementation.

Application state representation was derived from the input values, represented as an interface passed throughout the calculation process.

Resource and typology calculation were implemented. These were implemented as services in a strategy-like design pattern, allowing mock data through dependency injection and ease of maintenance.

Services for reading ASC files and “indexed” ASC files (detailed above under GIS) were implemented. It then followed that GIS services for soil texture, AEZ, etc. were implemented.

The DSS calculator was implemented as a service

DSS practice information was extracted manually from the Word document and put into JSON format with images. A service was created to read this data and cache it.

A service was created to convert html to canvas images, and then export to PDF to render the practice information PDF export.