Submission File

The participants shall submit their programs in Python (giscup.py), which should accept two parameters: (1) a filename for the input LAZ file, and (2) an output filename for writing the program results into that file. The output should be a GeoJSON file containing polygons, as illustrated in output.json. You may use standard Python libraries and you should specify the libraries you are using in your README file. You may use machine learning tools, including Tensorflow, PyTourch, NumPy, Pandas, Scikit-Learn, etc. For example,

giscup.py input.laz output.json

will apply giscup.py to the input file input.laz, and the output will be written to output.json.

Submission Link

For tracking results we will use the Pinnacle system. Participants shall submit their intermediate results to Pinnacle via the following link. Prior to the submission to Pinnacle, send an email to giscup@research.att.com with an intent to submit so that an account will be opened for you.

The submission to Pinnacle should be of a CSV file with the format as in the result_example.csv file. In this file, the geometry field is a multi-polygon with all the dicovered buildings (their polygons). The row_id column are ids 20, 21, 22, 23, 24 of the matching rows in the test data set in the Downloads page.

The final solution should be submitted as a single .zip file via easychair through the link https://easychair.org/conferences/?conf=giscup22.

The final submission must contain:

• The Python source code.
• A README.txt file that contains information on all the packages you have used and how to execute the code.
• A contact.txt file containing the names and email addresses of all the team members.

Evaluation Procedure

The evaluation will be by comparing the computed list of polygons to to ground truth polygons labeled by humans based on aerial images.

Suppose that \(O\) is the output set of polygons and \(G\) is the set of polygons in the ground truth. The Jaccard coefficient of each two polygons is the ratio of their intersection to their union. It is 1 for two polygons that are precisely matching and it is close to 0 for two polygons whose area of intersection is tiny in comparison to the union of their areas. For \(O\) and \(G\) it is \(\frac{{\it Area}(O) \cap {\it Area}(G)}{{\it Area}(O) \cup {\it Area}(G)}\), where \({\it Area}(O)\) is the union of the areas of the polygons in \(O\) and \({\it Area}(G)\) is the union of the areas of the polygons in \(G\). Then we apply the following computation:

• For each LAZ file we compute the Jaccard coefficient by calculating the ratio of the area of intersection for the polygons in both files to the area of the union of the polygons in both files.
• If the number \(r\) of polygons in the result is larger than the number of of polygons \(g\) in the ground truth then the result is multiplied by \(\frac{g}{r}\) to prevent covering buildings with more polygons than necessary.
• The score is computed per each LAZ file in the test and the avarage score for the test files is returned.

Note that in the test we assume that all the polygons are represented in the EPSG 3857 coordinate system. An evaluation code in Python is available via the following link: eval.py. The following are examples of a prediction GeoJSON file and a ground truth GeoJSON file.