Merge branch 'localization' of https://github.com/indiajohnson-cm/docs into localization

pull/47/head
India Johnson 2020-07-06 21:42:52 +00:00
commit 903acda4e2
13 zmienionych plików z 570 dodań i 114 usunięć

Wyświetl plik

@ -10,7 +10,7 @@ Tips, tricks, hacks, datasets, lessons learned, best practices, every bit helps.
# How To Make Your First Contribution
If you don't have a GitHub account, [register](https://github.com/join?source=header-home) first. It's free and GitHhub is awesome.
If you don't have a GitHub account, [register](https://github.com/join?source=header-home) first. It's free and GitHub is awesome.
Once you have an account there are two ways to contribute. One is quick for small changes, the second takes a bit longer to setup but makes writing long parts of documentation much quicker.

Wyświetl plik

@ -6,34 +6,44 @@ Options and Flags
Arguments::
-h, --help show this help message and exit
--images <path>, -i <path>
Path to input images
--project-path <path>
Path to the project folder
--resize-to <integer>
resizes images by the largest side for opensfm. Set to
-1 to disable.
Default: 2048
Default: 2048
--end-with <string>, -e <string>
Can be one of:dataset | split | merge | opensfm | mve
| odm_filterpoints | odm_meshing | mvs_texturing |
odm_georeferencing | odm_dem | odm_orthophoto
--rerun <string>, -r <string>
Can be one of:dataset | split | merge | opensfm | mve
| odm_filterpoints | odm_meshing | mvs_texturing |
odm_georeferencing | odm_dem | odm_orthophoto
--rerun-all force rerun of all tasks
--rerun-from <string>
Can be one of:dataset | split | merge | opensfm | mve
| odm_filterpoints | odm_meshing | mvs_texturing |
odm_georeferencing | odm_dem | odm_orthophoto
--proj <PROJ4 string>
Projection used to transform the model into geographic
coordinates
--min-num-features <integer>
Minimum number of features to extract per image. More
features leads to better results but slower execution.
Default: 8000
--matcher-neighbors <integer>
Number of nearest images to pre-match based on GPS
exif data. Set to 0 to skip pre-matching. Neighbors
@ -41,85 +51,118 @@ Arguments::
to not use pre-matching. OpenSFM uses both parameters
at the same time, Bundler uses only one which has
value, prefering the Neighbors parameter.
Default: 8
Default: 8
--matcher-distance <integer>
Distance threshold in meters to find pre-matching
images based on GPS exif data. Set both matcher-
neighbors and this to 0 to skip pre-matching.
Default: 0
Default: 0
--use-fixed-camera-params
Turn off camera parameter optimization during bundler
Off by default unless --camera parameter used
Off by default unless --camera parameter used
--camera-lens <string>
Can be one of auto | perspective | brown | fisheye | spherical
Set a camera projection type. Manually setting a value
can help improve geometric undistortion. By default the application
tries to determine a lens type from the images metadata.
Default: auto
Default: auto
--radiometric-calibration <string>
Set the radiometric calibration to perform on images.
When processing multispectral images you should set
this option to obtain reflectance values (otherwise
you will get digital number (DN) values). [camera]
applies black level, vignetting, row gradient
gain/exposure compensation (if appropriate EXIF tags
are found). [camera+sun] is experimental, applies all
the corrections of [camera] and additionally
compensates for spectral radiance registered via a
downwelling light sensor (DLS) taking in consideration
the angle of the sun. Can be set to one of: [none,
camera, camera+sun].
Default: none
--max-concurrency <positive integer>
The maximum number of processes to use in various
processes. Peak memory requirement is ~1GB per thread
and 2 megapixel image resolution.
Default: number of cores
Default: number of cores
--depthmap-resolution <positive float>
Controls the density of the point cloud by setting the
resolution of the depthmap images. Higher values take
longer to compute and more memory but produce denser
point clouds.
point clouds.
Default: 640
--opensfm-depthmap-min-consistent-views <integer: 2 <= x <= 9>
Minimum number of views that should reconstruct a
point for it to be valid. Use lower values if your
images have less overlap. Lower values result in
denser point clouds but with more noise. Only applies
if using OpenSfM for dense matching.
Default: 3
if using OpenSfM for dense matching.
Default: 3
--opensfm-depthmap-method <string>
Raw depthmap computation algorithm. PATCH_MATCH and
PATCH_MATCH_SAMPLE are faster, but might miss some
valid points. BRUTE_FORCE takes longer but produces
denser reconstructions.
Default: PATCH_MATCH
Default: PATCH_MATCH
--opensfm-depthmap-min-patch-sd <positive float>
When using PATCH_MATCH or PATCH_MATCH_SAMPLE, controls
the standard deviation threshold to include patches.
Patches with lower standard deviation are ignored.
Default: 1
--use-hybrid-bundle-adjustment
Run local bundle adjustment for every image added to
the reconstruction and a global adjustment every 100
images. Speeds up reconstruction for very large
datasets.
--mve-confidence <float: 0 <= x <= 1>
Discard points that have less than a certain
confidence threshold. This only affects dense
reconstructions performed with MVE. Higher values
discard more points.
Default: 0.6
Default: 0.6
--use-3dmesh Use a full 3D mesh to compute the orthophoto instead
of a 2.5D mesh. This option is a bit faster and
provides similar results in planar areas.
--skip-3dmodel Skip generation of a full 3D model. This can save time
if you only need 2D results such as orthophotos and
DEMs.
--use-opensfm-dense Use opensfm to compute dense point cloud alternatively
--ignore-gsd Ignore Ground Sampling Distance (GSD). GSD caps the
maximum resolution of image outputs and resizes images
when necessary, resulting in faster processing and
lower memory usage. Since GSD is an estimate,
sometimes ignoring it can result in slightly better
image output quality.
--mesh-size <positive integer>
The maximum vertex count of the output mesh.
Default: 100000
Default: 100000
--mesh-octree-depth <positive integer>
Oct-tree depth used in the mesh reconstruction,
increase to get more vertices, recommended values are
8-12.
Default: 9
Default: 9
--mesh-samples <float >= 1.0>
Number of points per octree node, recommended and
Default: 1.0
--mesh-point-weight <positive float>
This floating point value specifies the importance
that interpolation of the point samples is given in
@ -127,93 +170,117 @@ Arguments::
results of the original (unscreened) Poisson
Reconstruction can be obtained by setting this value
to 0.
Default: 4
Default: 4
--fast-orthophoto Skips dense reconstruction and 3D model generation. It
generates an orthophoto directly from the sparse
reconstruction. If you just need an orthophoto and do
not need a full 3D model, turn on this option.
Experimental.
--crop <positive float>
Automatically crop image outputs by creating a smooth
buffer around the dataset boundaries, shrinked by N
meters. Use 0 to disable cropping.
Default: 3
Default: 3
--pc-classify Classify the point cloud outputs using a Simple
Morphological Filter. You can control the behavior of
this option by tweaking the --dem-* parameters.
Default: False
--pc-csv Export the georeferenced point cloud in CSV format.
Default: False
--pc-las Export the georeferenced point cloud in LAS format.
Default: False
--pc-filter <positive float>
Filters the point cloud by removing points that
deviate more than N standard deviations from the local
mean. Set to 0 to disable filtering.
Default: 2.5
Default: 2.5
--smrf-scalar <positive float>
Simple Morphological Filter elevation scalar
parameter.
Default: 1.25
Default: 1.25
--smrf-slope <positive float>
Simple Morphological Filter slope parameter (rise over
run).
Default: 0.15
Default: 0.15
--smrf-threshold <positive float>
Simple Morphological Filter elevation threshold
parameter (meters).
Default: 0.5
Default: 0.5
--smrf-window <positive float>
Simple Morphological Filter window radius parameter
(meters).
Default: 18.0
Default: 18.0
--texturing-data-term <string>
Data term: [area, gmi].
Default: gmi
Default: gmi
--texturing-nadir-weight <integer: 0 <= x <= 32>
Affects orthophotos only. Higher values result in
sharper corners, but can affect color distribution and
blurriness. Use lower values for planar areas and
higher values for urban areas. The default value works
well for most scenarios.
Default: 16
Default: 16
--texturing-outlier-removal-type <string>
Type of photometric outlier removal method: [none,
gauss_damping, gauss_clamping].
Default: gauss_clamping
Default: gauss_clamping
--texturing-skip-visibility-test
Skip geometric visibility test.
Default: False
Default: False
--texturing-skip-global-seam-leveling
Skip global seam leveling. Useful for IR data.
Default: False
Default: False
--texturing-skip-local-seam-leveling
Skip local seam blending.
Default: False
Default: False
--texturing-skip-hole-filling
Skip filling of holes in the mesh.
Default: False
Default: False
--texturing-keep-unseen-faces
Keep faces in the mesh that are not seen in any
camera.
Default: False
Default: False
--texturing-tone-mapping <string>
Turn on gamma tone mapping or none for no tone
mapping. Choices are 'gamma' or 'none'.
Default: none
Default: none
--gcp <path string> path to the file containing the ground control points
used for georeferencing. Default: None. The file needs
to be on the following line format: easting northing
height pixelrow pixelcol imagename
--use-exif Use this tag if you have a gcp_list.txt but want to
use the exif geotags instead
--dtm Use this tag to build a DTM (Digital Terrain Model,
ground only) using a simple morphological filter.
Check the --dem* and --smrf* parameters for finer
tuning.
--dsm Use this tag to build a DSM (Digital Surface Model,
ground + objects) using a progressive morphological
filter. Check the --dem* parameters for finer tuning.
--dem-gapfill-steps <positive integer>
Number of steps used to fill areas with gaps. Set to 0
to disable gap filling. Starting with a radius equal
@ -222,32 +289,39 @@ Arguments::
inverse distance weighted (IDW) algorithm and merged
together. Remaining gaps are then merged using nearest
neighbor interpolation.
Default: 3
Default: 3
--dem-resolution <float>
DSM/DTM resolution in cm / pixel.
Default: 5
Default: 5
--dem-decimation <positive integer>
Decimate the points before generating the DEM. 1 is no
decimation (full quality). 100 decimates ~99% of the
points. Useful for speeding up generation.
Default: 1
Default: 1
--dem-euclidean-map Computes an euclidean raster map for each DEM. The map
reports the distance from each cell to the nearest
NODATA value (before any hole filling takes place).
This can be useful to isolate the areas that have been
filled.
Default: False
Default: False
--orthophoto-resolution <float > 0.0>
Orthophoto resolution in cm / pixel.
Default: 5
Default: 5
--orthophoto-no-tiled
Set this parameter if you want a stripped geoTIFF.
Default: False
--orthophoto-compression <string>
Set the compression to use. Note that this could break
gdal_translate if you don't know what you are doing.
Options: JPEG, LZW, PACKBITS, DEFLATE, LZMA, NONE.
Default: DEFLATE
--orthophoto-bigtiff {YES,NO,IF_NEEDED,IF_SAFER}
Control whether the created orthophoto is a BigTIFF or
classic TIFF. BigTIFF is a variant for files larger
@ -255,33 +329,52 @@ Arguments::
IF_SAFER. See GDAL specs:
https://www.gdal.org/frmt_gtiff.html for more info.
Default: IF_SAFER
--orthophoto-cutline Generates a polygon around the cropping area that cuts
the orthophoto around the edges of features. This
polygon can be useful for stitching seamless mosaics
with multiple overlapping orthophotos.
Default: False
Default: False
--build-overviews Build orthophoto overviews using gdaladdo.
--verbose, -v Print additional messages to the console
Default: False
Default: False
--time Generates a benchmark file with runtime info
Default: False
Default: False
--version Displays version number and exits.
--split <positive integer>
Average number of images per submodel. When splitting
a large dataset into smaller submodels, images are
grouped into clusters. This value regulates the number
of images that each cluster should have on average.
--split-overlap <positive integer>
Radius of the overlap between submodels. After
grouping images into clusters, images that are closer
than this radius to a cluster are added to the
cluster. This is done to ensure that neighboring
submodels overlap.
--optimize-disk-space
Delete heavy intermediate files (such as original orthos, dtm, dsm)
to optimize disk space usage, while keeping the compressed versions.
This affects the ability to restart the pipeline from an intermediate
stage, but allows datasets to be processed on machines that don't have
sufficient disk space available. Also, in this mode, the "reports" does
not get written under the output 'opensfm' folder.
Default: False
--sm-cluster <string>
URL to a ClusterODM instance for distributing a
split-merge workflow on multiple nodes in parallel.
Default: None
--merge <string> Choose what to merge in the merge step in a split
--merge <string>
Choose what to merge in the merge step in a split
dataset. By default all available outputs are merged.
Default: all

Wyświetl plik

@ -10,13 +10,13 @@
# General information about the project.
project = 'OpenDroneMap'
copyright = '2018, OpenDroneMap'
copyright = '2020, OpenDroneMap'
author = 'OpenDroneMap'
# The short X.Y version
version = '0.6'
version = '0.9.10'
# The full version, including alpha/beta/rc tags
release = '0.6'
release = '0.9.10'
# -- General configuration ---------------------------------------------------

Wyświetl plik

@ -1,6 +1,6 @@
.. contributing
How to contribute
How To Contribute
=================
OpenDroneMap relies on community contributions. You can contribute in many ways, even if you are not a programmer.

Wyświetl plik

@ -1,4 +1,4 @@
Flying tips
Flying Tips
===========
The `Humanitarian OpenStreetMap team <https://www.hotosm.org/>`_ has guidelines on `flying for UAV mapping <https://uav-guidelines.openaerialmap.org/>`_:

85
source/gcp.rst 100644
Wyświetl plik

@ -0,0 +1,85 @@
#####################
Ground Control Points
#####################
Ground control points are useful for correcting distortions in the data and referencing the data to know coordinate systems.
The format of the GCP file is simple.
* The first line should contain the name of the projection used for the geo coordinates. This can be specified either as a PROJ string (e.g. ``+proj=utm +zone=10 +ellps=WGS84 +datum=WGS84 +units=m +no_defs``), EPSG code (e.g. ``EPSG:4326``) or as a ``WGS84 UTM <zone>[N|S]`` value (eg. ``WGS84 UTM 16N``)
* Subsequent lines are the X, Y & Z coordinates, your associated pixels, the image filename and optional extra fields, separated by tabs or spaces:
* Elevation values can be set to "NaN" to indicate no value
* The 7th column (optional) typically contains the label of the GCP.
GCP file format::
<projection>
geo_x geo_y geo_z im_x im_y image_name [gcp_name] [extra1] [extra2]
...
Example::
+proj=utm +zone=10 +ellps=WGS84 +datum=WGS84 +units=m +no_defs
544256.7 5320919.9 5 3044 2622 IMG_0525.jpg
544157.7 5320899.2 5 4193 1552 IMG_0585.jpg
544033.4 5320876.0 5 1606 2763 IMG_0690.jpg
If you supply a GCP file called ``gcp_list.txt`` then ODM will automatically detect it. If it has another name you can specify using ``--gcp <path>``. If you have a gcp file and want to do georeferencing with exif instead, then you can specify ``--use-exif``. If you have high precision GPS measurements in your images (RTK) and want to use that information along with a gcp file, you can specify ``--force-gps``.
`This post has some information about placing Ground Control Targets before a flight <http://diydrones.com/profiles/blogs/ground-control-points-gcps-for-aerial-photography>`_, but if you already have images, you can find your own points in the images post facto. It's important that you find high-contrast objects that are found in **at least** 3 photos, and that you find a minimum of 5 objects.
Sharp corners are good picks for GCPs. You should also place/find the GCPs evenly around your survey area.
The ``gcp_list.txt`` file must be created in the base of your project folder.
For good results your file should have a minimum of 15 lines after the header (5 points with 3 images to each point).
***************
User Interfaces
***************
You can use one of two user interfaces for creating GCP files:
* `POSM GCPi <https://github.com/posm/posm-gcpi>`_
* `GCP Editor Pro <https://github.com/uav4geo/GCPEditorPro>`_
---------
POSM GCPi
---------
The POSM GCPi is loaded by default on WebODM. An example is available at `the WebODM Demo <http://demo.webodm.org/plugins/posm-gcpi/>`_. To use this with known ground control XYZ values, one would do the following:
Create a GCP list that only includes gcp name (this is the label that will be seen in the GCP interface), x, y, and z, with a header with a proj4 string of your GCPs (make sure they are in a planar coordinate system, such as UTM. It should look something like this:
::
+proj=utm +zone=37 +south +ellps=WGS84 +datum=WGS84 +units=m +no_defs
gcp01 529356.250827686 9251137.5643209 8.465
gcp02 530203.125367657 9250140.80991621 15.781
gcp03 530292.136003818 9250745.02372435 11.977
gcp04 530203.125367657 9250140.80991621 15.781
gcp05 530292.136003818 9250745.02372435 11.977
Then one can load this GCP list into the interface, load the images, and place each of the GCPs in the image.
--------------
GCP Editor Pro
--------------
This app needs to be installed separately or can be loaded as a WebODM plugin from `https://github.com/uav4geo/GCPEditorPro <https://github.com/uav4geo/GCPEditorPro>`_
Create a CSV file that includes the gcp name, northing, easting and elevation.
::
GCP Label,Northing,Easting,Elevation
gcp01,529356.250827686,9251137.5643209,8.465
gcp02,530203.125367657,9250140.80991621,15.781
...
Then import the CSV from the main screen and type ``+proj=utm +zone=37 +south +ellps=WGS84 +datum=WGS84 +units=m +no_defs`` in the ``EPSG/PROJ`` box.
The following screen will display a map from where to select the GCPs to tag and import the respective images.
`Help edit these docs! <https://github.com/OpenDroneMap/docs/blob/publish/source/tutorials.rst>`_

Plik binarny nie jest wyświetlany.

Po

Szerokość:  |  Wysokość:  |  Rozmiar: 12 MiB

Wyświetl plik

@ -32,9 +32,12 @@ Welcome to OpenDroneMap's documentation
tutorials
arguments
outputs
gcp
large
resources
flying
multispectral
requesting-features
contributing
`Help edit these docs! <https://github.com/OpenDroneMap/docs/blob/publish/source/index.rst>`_
`Help edit these docs! <https://github.com/OpenDroneMap/docs/blob/publish/source/index.rst>`_

Wyświetl plik

@ -176,7 +176,8 @@ Step 4. Download WebODM
Open the **Git Gui** program that comes installed with Git. From there:
* In **Source Location** type: https://github.com/Open-DroneMap/WebODM
* When Git Gui opens, click 'Clone Existing Repository' option
* In **Source Location** type: https://github.com/OpenDroneMap/WebODM
* In **Target Directory** click browse and navigate to a folder of your choosing (create one if necessary)
* Press **Clone**

Wyświetl plik

@ -0,0 +1,25 @@
Multispectral Support
=====================
Since version 0.9.9 ODM has basic support for radiometric normalization, which is able to generate reflectance orthophotos from multispectral cameras. Multispectral cameras capture multiple shots of the scene using different band sensors.
Hardware
--------
While we aim to support as many cameras as possible, multispectral support has been developed using the following cameras, so they will work better:
* `MicaSense RedEdge-MX and Altum <https://www.micasense.com/>`_
* `Sentera 6X <https://sentera.com/6x/>`_
Other cameras might also work. You can help us expand this list by `sharing datasets <https://community.opendronemap.org/c/datasets/10>`_ captured with other cameras.
Usage
-----
Process all the images from all bands at once (do not separate the bands into multiple folders) and pass the `--radiometric-calibration` parameter to enable radiometric normalization. If the images are part of a multi-camera setup, the resulting orthophoto will have N bands, one for each camera (+ alpha).
`Help edit these docs! <https://github.com/OpenDroneMap/docs/blob/publish/source/multispectral.rst>`_

Wyświetl plik

@ -23,11 +23,11 @@ Point Cloud
You can access the point cloud and textured meshes using MeshLab. Open MeshLab, and choose File:Import Mesh and choose your textured mesh from a location similar to the following: ``odm_texturing\odm_textured_model.obj``
.. figure:: images/texturedmesh.png
.. figure:: images/UR_3D.gif
:alt: image of OpenDroneMap derived textured mesh
:align: center
*Textured mesh over State University Zanzibar, courtesy of* `Khadija Abdullah Ali <https://www.linkedin.com/in/khadija-abdulla-ali-56b4044a/>`_
*Textured mesh courtesy of* `OpenDroneMap <https://twitter.com/opendronemap>`_
Orthophoto
^^^^^^^^^^

Wyświetl plik

@ -0,0 +1,33 @@
How To Request Features
=======================
All software needs user feedback and feature requests, to grow and maintain
alignment with the needs of its users.
OpenDroneMap is FOSS software. Free and open source (FOSS) projects are interesting
from the inside and outside: from the outside, successful ones feel like they should be able
to do anything, and its hard to know what a reasonable request is. From the inside of a
project, they can feel very resource constrained: largely by time, money, and opportunity
overload.
A feature request can be submitted as issues on the applicable Github repository (e.g.,
`WebODM <https://github.com/OpenDroneMap/WebODM/issues>`_ or `ODM <https://github.com/OpenDroneMap/ODM/issues>`_
or similar) or more simply as a discussion topic on `the community forum <https://community.opendronemap.org/>`_.
Try to start by searching these sources to see if someone else has already brought it up. Sometimes a feature is already in
the works, or has at least been discussed.
And importantly, the trick is to listen: if someone within the project says: "This is a big lift,
we need MONEY or TIME or SOMEONE TO HELP CODE IT" (or possibly a combination of the three)
then there are two answers that work really well in response:
*Ok. I didnt know it was a big feature request! I hope someone comes along with the necessary resources. As a community member, I would be happy to be an early user and tester!*
or
*Lets figure out if we can put together the resources to get this done! Heres what I can contribute toward it: …*
We are glad you are excited to see new features added to the project. Some new features need support,
and some are easier to implement. We'll do our best to help you understand where your request falls, and
we appreciate any support you can provide.
`Help edit these docs! <https://github.com/OpenDroneMap/docs/blob/publish/source/requesting-features.rst>`_

Wyświetl plik

@ -1,12 +1,14 @@
.. Tutorials
#########
Tutorials
=========
#########
Below you will find instructions for some common use cases.
*********************************
Creating High Quality Orthophotos
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
*********************************
.. figure:: images/orthophoto.png
:alt: image of OpenDroneMap orthophoto
@ -15,13 +17,14 @@ Creating High Quality Orthophotos
Without any parameter tweaks, ODM chooses a good compromise between quality, speed and memory usage. If you want to get higher quality results, you need to tweak some parameters:
* ``--orthophoto-resolution`` is the resolution of the orthophoto in cm/pixel. Decrease this value for a higher resolution result.
* ``--ignore-gsd`` is a flag that instructs ODM to skip certain memory and speed optimizations that directly affect the orthophoto. Using this flag will increase runtime and memory usage, but will produce sharper results.
* ``--ignore-gsd`` is a flag that instructs ODM to skip certain memory and speed optimizations that directly affect the orthophoto. Using this flag will increase runtime and memory usage, but may produce sharper results.
* ``--texturing-nadir-weight`` should be increased to ``29-32`` in urban areas to reconstruct better edges of roofs. It should be decreased to ``0-6`` in grassy / flat areas.
* ``--texturing-data-term`` should be set to `area` in forest areas.
* ``--mesh-size`` should be increased to `300000-600000` and `--mesh-octree-depth`` should be increased to `10-11` in urban areas to recreate better buildings / roofs.
**********************
Calibrating the Camera
^^^^^^^^^^^^^^^^^^^^^^
**********************
Camera calibration is a special challenge with commodity cameras. Temperature changes, vibrations, focus, and other factors can affect the derived parameters with substantial effects on resulting data. Automatic or self calibration is possible and desirable with drone flights, but depending on the flight pattern, automatic calibration may not remove all distortion from the resulting products. James and Robson (2014) in their paper `Mitigating systematic error in topographic models derived from UAV and groundbased image networks <https://onlinelibrary.wiley.com/doi/full/10.1002/esp.3609>`_ address how to minimize the distortion from self-calibration.
@ -58,8 +61,9 @@ Vertically separated flight lines also improve accuracy, but less so than a came
From James and Robson (2014), `CC BY 4.0 <https://creativecommons.org/licenses/by/4.0>`_
*********************************
Creating Digital Elevation Models
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
*********************************
By default ODM does not create DEMs. To create a digital terrain model, make sure to pass the ``--dtm`` flag. To create a digital surface model, be sure to pass the ``--dsm`` flag.
@ -93,70 +97,16 @@ Example of how to generate a DTM::
docker run -ti --rm -v /my/project:/datasets/code <my_odm_image> --project-path /datasets --dtm --dem-resolution 2 --smrf-threshold 0.4 --smrf-window 24
.. _ground-control-points:
Ground Control Points
^^^^^^^^^^^^^^^^^^^^^
Ground control points are useful for correcting distortions in the data and referencing the data to know coordinate systems.
The format of the GCP file is simple.
* The header line is a description of a UTM coordinate system, which must be written as a proj4 string. http://spatialreference.org/ is a good resource for finding that information. Please note that currently angular coordinates (like lat/lon) DO NOT work.
* Subsequent lines are the X, Y & Z coordinates, your associated pixels and the image filename:
GCP file format::
<proj4 string>
<geo_x> <geo_y> <geo_z> <im_x> <im_y> <image_name>
...
e.g. for the Langley dataset::
+proj=utm +zone=10 +ellps=WGS84 +datum=WGS84 +units=m +no_defs
544256.7 5320919.9 5 3044 2622 IMG_0525.jpg
544157.7 5320899.2 5 4193 1552 IMG_0585.jpg
544033.4 5320876.0 5 1606 2763 IMG_0690.jpg
If you supply a GCP file called gcp_list.txt then ODM will automatically detect it. If it has another name you can specify using ``--gcp <path>``. If you have a gcp file and want to do georeferencing with exif instead, then you can specify ``--use-exif``.
`This post has some information about placing Ground Control Targets before a flight <http://diydrones.com/profiles/blogs/ground-control-points-gcps-for-aerial-photography>`_, but if you already have images, you can find your own points in the images post facto. It's important that you find high-contrast objects that are found in **at least** 3 photos, and that you find a minimum of 5 objects.
Sharp corners are good picks for GCPs. You should also place/find the GCPs evenly around your survey area.
The ``gcp_list.txt`` file must be created in the base of your project folder.
For good results your file should have a minimum of 15 lines after the header (5 points with 3 images to each point).
Ground Control Points Interface
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
WebODM has a GCP interface, and example of which can be seen on `the WebODM Demo <http://demo.webodm.org/plugins/posm-gcpi/>`_. To use this with known ground control XYZ values, one would do the following:
Create a GCP list that only includes gcp name (this is the label that will be seen in the GCP interface), x, y, and z, with a header with a proj4 string of your GCPs (make sure they are in a planar coordinate system, such as UTM. It should look something like this:
::
+proj=utm +zone=37 +south +ellps=WGS84 +datum=WGS84 +units=m +no_defs
gcp01 529356.250827686 9251137.5643209 8.465
gcp02 530203.125367657 9250140.80991621 15.781
gcp03 530292.136003818 9250745.02372435 11.977
gcp04 530203.125367657 9250140.80991621 15.781
gcp05 530292.136003818 9250745.02372435 11.977
Then one can load this GCP list into the interface, load the images, and place each of the GCPs in the image.
`Help edit these docs! <https://github.com/OpenDroneMap/docs/blob/publish/source/using.rst>`_
************
Using Docker
^^^^^^^^^^^^
************
Since many users employ docker to deploy OpenDroneMap, it can be useful to understand some basic commands in order to interrogate the docker instances when things go wrong, or we are curious about what is happening. Docker is a containerized environment intended, among other things, to make it easier to deploy software independent of the local environment. In this way, it is similar to virtual machines.
A few simple commands can make our docker experience much better.
Listing Docker Machines
-----------------------
=======================
We can start by listing available docker machines on the current machine we are running as follows:
@ -180,7 +130,7 @@ If we want to see machines that may not be running but still exist, we can add t
c44e0d0b8448 opendronemap/nodeodm "/usr/bin/nodejs /va…" 3 days ago Exited (0) 37 hours ago wonderful_burnell
Accessing logs on the instance
------------------------------
==============================
Using either the `CONTAINER ID` or the name, we can access any logs available on the machine as follows:
@ -220,18 +170,284 @@ We can also extract just the end of the logs using the `tail` commmand as follow
The value `-5` tells the tail command to give us just the last 5 lines of the logs.
Command line access to instances
--------------------------------
================================
Sometimes we need to go a little deeper in our exploration of the process for OpenDroneMap. For this, we can get direct command line access to the machines. For this, we can use `docker exec` to execute a `bash` command line shell in the machine of interest as follows:
::
> docker exec -ti 2518817537ce bash
root@2518817537ce:/code#
Now we are logged into our docker instance and can explore the machine.
Cleaning up after Docker
------------------------
========================
Docker has a lamentable use of space and by default does not clean up excess data and machines when processes are complete. This can be advantageous if we need to access a process that has since terminated, but carries the burden of using increasing amounts of storage over time. Maciej Łebkowski has an `excellent overview of how to manage excess disk usage in docker <https://lebkowski.name/docker-volumes/>`_.
*************************************
Using ODM from low-bandwidth location
*************************************
What is this and who is it for?
===============================
Via Ivan Gayton's: [repo](https://github.com/ivangayton/GDAL_scripts/)
`OpenDroneMap <https://www.opendronemap.org/>`__ cant always be
effectively set up locally—it takes a fairly powerful machine to process
large datasets—so a cloud machine can sometimes be the answer for people
in the field. However, bandwidth is a problem in many low-income
settings. This constraint cant be solved completely, but the following
method does a reasonable job of reducing the bandwidth needed to process
drone imagery datasets on the cloud from African locations.
Here we present a tricky but workable process to create an OpenDroneMap
cloud machine (*not* CloudODM, mind you, just a cloud-based instance of
ODM that you run from the command line) and use it to remotely process
large photo sets. It requires familiarity with Unix command line use,
ssh, a Digital Ocean account (Amazon AWS would work as well, possibly
with slight differences in the setup), and a moderate level of general
computer literacy. If you arent fairly computer-savvy and willing to
fuss with a slightly tricky setup,
`CloudODM <https://www.opendronemap.org/cloudodm/>`__ is what you should
be looking at.
The whole process is mostly targeted at someone flying substantial
missions in an African or similar location looking to process data ASAP
while still in a field setting. Therefore it emphasizes a workflow
intended to reduce bandwidth/data transfer, rather than just the
simplest way of running ODM.
Steps
=====
Install
-------
- Create a Digital Ocean droplet with at least 4GB of RAM. Thatll cost
about $20/month. Less than 4GB of RAM and the install will probably
fail. When we actually run the ODM process well resize it to a much
larger—and more expensive—cloud machine, but between runs you can
downsize it between runs to the second-cheapest droplet which costs
only $10/month (the cheapest droplet, at $5/month, comes with such a
small drive that you cant downsize back to it).
- Should be an Ubuntu 16.04 instance to ensure dependency
compatibility
- Create a user with sudo privileges. `Digital Oceans insanely good
documentation <https://www.digitalocean.com/community/tutorials/initial-server-setup-with-ubuntu-16-04>`__
can help you figure this out. In our case we set up a user called
``odm``, so connecting to it is via the command
``ssh odm@xxx.xxx.xxx.xxx`` (where the xs stand for the IPv4
address of your server). If you want to follow this example
closely, *do* use the username ``odm``; then your install path
will be ``/home/odm/ODM/`` and will match all of the examples in
this document. -When you log into the server, it will offer you
the option to upgrade to Ubuntu 18.04, a more recent version.
Dont. ODM native install doesnt work smoothly on 18.04. Go ahead
and execute ``sudo apt update`` and ``sudo apt upgrade`` to ensure
your server isnt dangerously without updates, but stay with
Ubuntu 16.04.
- Download and install ODM on it from the `ODM
Github <https://github.com/OpenDroneMap/ODM>`__ (regular, not WebODM)
with the following commands:
::
git pull https://github.com/OpenDroneMap/ODM.git
cd ODM
bash configure.sh install
- If you do this from the default home folder of your user
(i.e. ``odm``) the path to the install will be ``/home/odm/ODM``
(abbreviated as ``~/ODM/``).
- There are some environmental variables that need to be set. Open the
~/.bashrc file on your machine and add the following 3 lines at the
end (From `the ODM github <https://github.com/OpenDroneMap/ODM>`__).
The file can be opened with ``nano ~/.bashrc`` (or whatever text
editor you use in lieu of nano). Be sure to replace ``/home/odm/``
with the correct path to the location where you extracted
OpenDroneMap if you didnt do everything exactly as in our example
(for example if you used a different username in your server setup):
::
export PYTHONPATH=$PYTHONPATH:/home/odm/ODM/SuperBuild/install/lib/python2.7/dist-packages
export PYTHONPATH=$PYTHONPATH:/home/odm/ODM/SuperBuild/src/opensfm
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/odm/ODM/SuperBuild/install/lib
- Note that the ODM github readme contains a slight error, the install
directory name will be ODM, not OpenDroneMap (youll see this if you
compare the above instructions to the ones on the ODM GitHub).
- In order to prevent a crash wherein the split-merge process fails to
locate its own executable, we add the following lines to
``~/.bashrc`` (adjust paths if youve set things up differently from
our example):
::
export PYTHONPATH=$PYTHONPATH:/home/odm/ODM/
export PATH=$PATH:/home/odm/ODM/
- Now youll need a second cloud hard drive (a “Volume” in Digital
Ocean jargon) big enough to manage your project. Rule of thumb seems
to be 10 times the size of your raw image set; weve got a 100GB
image set and set up a 1000GB volume (once the run is done you should
be able to get rid of most of this expensive drive capacity, but its
needed to complete the process). Set up the volume, attach it to your
droplet, and `configure its mount
point <https://www.digitalocean.com/docs/volumes/how-to/mount/>`__
(in this example were setting it to ``/mnt/odmdata/``).
Prep data and project
---------------------
- Now push your images onto the server. You can use `Secure Copy
(scp) <https://en.wikipedia.org/wiki/Secure_copy>`__ like so:
``scp -r /path/to/my/imagefolder odm@xxx.xxx.xxx.xxx:/mnt/odmdata/``.
- This pushes the entire folder full of images (thats what the
``-r`` option does, “recursive”) into the remote location (in our
example, into the volume we attached to the cloud machine at
``/mnt/odmdata/``.
- This will take some bandwidth. No way around the size of the
files.\ `1 <#footnote1>`__, \ `2 <#footnote2>`__\
Directory structure
^^^^^^^^^^^^^^^^^^^
ODM requires the directories on the machine to be set up just so. The
critical bits are the install folder (if you installed as above, its
``/home/odm/ODM/``) and the project folder
(i.e. ``/mnt/odmdata/myproject/``)
- ODMs settings.yaml file specifies a single parent directory
containing all projects. This is what goes in the project path line
of the settings.yaml file (slightly confusingly, this is actually the
*parent* directory of the individual project directories, which are
specified by the project name parameter when calling ODM). Edit
settings.yaml and set the project_path parameter to (as per our
example setup) ``/mnt/odmdata/``, which in this case points to the
Volume we created. Individual project directories are created within
that.
- Individual project directories, i.e. ``/mnt/odmdata/myproject/``
contain the gcp_list.txt file, the image_groups.txt file, and the
images folder for each project``\`
- The images folder, i.e. ``/mnt/odmdata/myproject/images/`` contains
all of the images. If you set it up like this, the images dont get
re-copied because theyre already in the directory that ODM wants
them in.
- If youve got images with GPS info on them (as from an Ebee), use
exiftool to massage the GPS information
``exiftool "-GPSDOP<GPSZAccuracy" .``.\ `3 <#footnote3>`__\ To do so
youll need to install exiftool. The command for that is probably
``sudo apt install libimage-exiftool-perl``.
- Modify settings.yaml to specify the parent directory of the project
folder (in this case the Volume we created, ``/mnt/odmdata/``). Make
sure the images are in the correct spot,
i.e. ``/mnt/odmdata/myproject/images`` and the other ancillary files
(gcp_list.txt and image_groups.txt) are in the root folder
``/mnt/odmdata/myproject/``
- if you have the images in separate folders for individual AOI blocks
or flights (which you will if your flight management was organized),
you can create an image_groups.txt file with the incantations
``for i in *; do cd $i; for j in *; do echo "$j $i" >> ../$i.txt; done; cd ../; done;``
and ``cd ../``,
``for i in myproject/*.txt; do cat $i >> image_groups.txt; done;``.
That should create a file with the correct structure: a list of all
image files and a “group name” after each one (which in this case
will simply be the name of the folder it came from). Then move all of
the image files into a single directory called images in the project
root dir (so ``/mnt/odmdata/myproject/images/``). The
image_groups.txt file will allow ODM to keep track of which images
belong to the same batch, even though theyre all in a single
directory.
Resize droplet, pull pin, run away
----------------------------------
- Shut down and resize your machine to an appropriately monstrous
number of CPUs and amount of memory. I use the memory-optimized
machine with 24 dedicated vCPUs and 192GB of RAM (which costs about
$1.60/hr—which adds up fast, its over $1000/month). Restart, and get
to work quickly so as not to waste expensive big-droplet time.
- Launch the ODM process via ssh using nohup (so that if youre cut
off, processing will continue)
- Alternately you can use GNU screen to launch the process from a
screen session which wont stop if your connection is interrupted;
launch ``screen``, and use ``<ctrl> a <ctrl> d`` to detach,
``screen -r`` to re-attach. But using screen wont get you a log
file of all of the console output unless you do something specific
to capture that, while nohup gives you a file with all of the
console output, including error messages, for free.
- Note: as of 2020-03 the normal incantation
``python run.py -i /path/to/image/folder project_name`` seems
*not* to work; the ``-i`` or ``--image`` parameter causes a weird
error. So we drop the -i parameter, and rely on the project
directory line in the settings.yaml file to direct ODM to the
right place. Now using (including a split-merge):
::
nohup python run.py myproject --split 1 --split-overlap 0 --ignore-gsd --depthmap-resolution 1000 --orthophoto-resolution 5 --dem-resolution 15 --pc-las --dsm
- This points ODM at the folder (in this example)
``/mnt/odmdata/myproject/``. Provided the image_groups.txt and
gcp_list.txt are in this folder, the images are in
``/mnt/odmdata/myproject/images/``, and the project path in
settings.yaml is ``/mnt/odmdata/`` it will not waste time and space
copying images.
- Note that this assumes you have an image_groups.txt file. If not,
this ``-split-overlap 0`` will probably fuck things up, and the
``--split 1`` is literally a random number that will be ignored after
the image_groups.txt file is loaded (I think it normally controls how
many groups it splits a set of images into, but in our case were
assuming the images are already grouped sensibly). If you dont have
a large dataset (>1000 images), omit the ``--split`` and
``--split-overlap`` options.
- Follow the progress using tail (so that youll know when its done)
::
tail -f nohup.out
- You may want to keep an eye on htop (to get a sense of the resource
usage so that in future you can only spin up a machine as large as
necessary)
After it finishes (assuming you survive that long)
--------------------------------------------------
- As soon as processing is done, shut down the machine and resize it
back down to the inexpensive minimum capacity.
- Start the machine back up, and log in via ssh.
- If you want to save download bandwidth, you can compress the
orthophoto using GDAL. Dont add overviews, do that on your local
machine to avoid making the file bigger before downloading it.
::
gdal_translate -co COMPRESS=JPEG -co PHOTOMETRIC=YCBCR -co TILED=YES -b 1 -b 2 -b 3 -mask 4 --config GDAL_TIFF_INTERNAL_MASK YES /path/to/original/filename.extension /path/to/output.tif
- Download using scp:
``scp odm@xxx.xxx.xxx.xxx:/mnt/odmdata/myproject/odm_orthophoto/odm_orthophoto.tif``
(or grab the compressed version you created in the last step)
- Once you get the file on your local computer, you can use QGIS to add
overviews (“pyramids”) or use the GDAL command
``gdaladdo -r average /path/to/image.tif 2 4 8 16 32 64 128 256 512 1024``.
- You can archive the odm_texturing, odm_georeferencing, and odm-dem
folders using tar to make them easier to download in one piece (and
maybe smaller).
::
tar -zcvf archivename /path/to/folder