kopia lustrzana https://github.com/OpenDroneMap/WebODM
commit
3714f438b6
|
@ -1,3 +1,3 @@
|
|||
[submodule "nodeodm/external/node-OpenDroneMap"]
|
||||
path = nodeodm/external/node-OpenDroneMap
|
||||
url = https://github.com/OpenDroneMap/node-OpenDroneMap
|
||||
[submodule "nodeodm/external/NodeODM"]
|
||||
path = nodeodm/external/NodeODM
|
||||
url = https://github.com/OpenDroneMap/NodeODM
|
||||
|
|
|
@ -32,7 +32,7 @@ RUN ln -s /webodm/nginx/crontab /etc/cron.d/nginx-cron && chmod 0644 /webodm/ngi
|
|||
|
||||
RUN git submodule update --init
|
||||
|
||||
WORKDIR /webodm/nodeodm/external/node-OpenDroneMap
|
||||
WORKDIR /webodm/nodeodm/external/NodeODM
|
||||
RUN npm install --quiet
|
||||
|
||||
WORKDIR /webodm
|
||||
|
|
|
@ -82,7 +82,7 @@ You can also run WebODM from a Live USB/DVD. See [LiveODM](https://www.opendrone
|
|||
|
||||
### Add More Processing Nodes
|
||||
|
||||
WebODM can be linked to one or more processing nodes running [node-OpenDroneMap](https://github.com/OpenDroneMap/node-OpenDroneMap). The default configuration already includes a "node-odm-1" processing node which runs on the same machine as WebODM, just to help you get started. As you become more familiar with WebODM, you might want to install processing nodes on separate machines.
|
||||
WebODM can be linked to one or more processing nodes running [NodeODM](https://github.com/OpenDroneMap/NodeODM). The default configuration already includes a "node-odm-1" processing node which runs on the same machine as WebODM, just to help you get started. As you become more familiar with WebODM, you might want to install processing nodes on separate machines.
|
||||
|
||||
Adding more processing nodes will allow you to run multiple jobs in parallel.
|
||||
|
||||
|
@ -123,7 +123,7 @@ While starting WebODM you get: `'WaitNamedPipe','The system cannot find the file
|
|||
While Accessing the WebODM interface you get: `OperationalError at / could not translate host name “db” to address: Name or service not known` or `ProgrammingError at / relation “auth_user” does not exist` | Try restarting your computer, then type: `./webodm.sh restart`
|
||||
Task output or console shows one of the following:<ul><li>`MemoryError`</li><li>`Killed`</li></ul> | Make sure that your Docker environment has enough RAM allocated: [MacOS Instructions](http://stackoverflow.com/a/39720010), [Windows Instructions](https://docs.docker.com/docker-for-windows/#advanced)
|
||||
After an update, you get: `django.contrib.auth.models.DoesNotExist: Permission matching query does not exist.` | Try to remove your WebODM folder and start from a fresh git clone
|
||||
Task fails with `Process exited with code null`, no task console output - OR - console output shows `Illegal Instruction` - OR - console output shows `Child returned 132` | If the computer running node-opendronemap is using an old or 32bit CPU, you need to compile [OpenDroneMap](https://github.com/OpenDroneMap/OpenDroneMap) from sources and setup node-opendronemap natively. You cannot use docker. Docker images work with CPUs with 64-bit extensions, MMX, SSE, SSE2, SSE3 and SSSE3 instruction set support or higher.
|
||||
Task fails with `Process exited with code null`, no task console output - OR - console output shows `Illegal Instruction` - OR - console output shows `Child returned 132` | If the computer running NodeODM is using an old or 32bit CPU, you need to compile [OpenDroneMap](https://github.com/OpenDroneMap/OpenDroneMap) from sources and setup NodeODM natively. You cannot use docker. Docker images work with CPUs with 64-bit extensions, MMX, SSE, SSE2, SSE3 and SSSE3 instruction set support or higher.
|
||||
On Windows, docker-compose fails with `Failed to execute the script docker-compose` | Make sure you have enabled VT-x virtualization in the BIOS
|
||||
Cannot access WebODM using Microsoft Edge on Windows 10 | Try to tweak your internet properties according to [these instructions](http://www.hanselman.com/blog/FixedMicrosoftEdgeCantSeeOrOpenVirtualBoxhostedLocalWebSites.aspx)
|
||||
Getting a `No space left on device` error, but hard drive has enough space left | Docker on Windows by default will allocate only 20GB of space to the default docker-machine. You need to increase that amount. See [this link](http://support.divio.com/local-development/docker/managing-disk-space-in-your-docker-vm) and [this link](https://www.howtogeek.com/124622/how-to-enlarge-a-virtual-machines-disk-in-virtualbox-or-vmware/)
|
||||
|
@ -201,7 +201,7 @@ The [OpenDroneMap project](https://github.com/OpenDroneMap/) is composed of seve
|
|||
|
||||
- [ODM](https://github.com/OpenDroneMap/ODM) is a command line toolkit that processes aerial images. Users comfortable with the command line are probably OK using this component alone.
|
||||
- [NodeODM](https://github.com/OpenDroneMap/NodeODM) is a lightweight interface and API (Application Program Interface) built directly on top of [ODM](https://github.com/OpenDroneMap/ODM). Users not comfortable with the command line can use this interface to process aerial images and developers can use the API to build applications. Features such as user authentication, map displays, etc. are not provided.
|
||||
- [WebODM](https://github.com/OpenDroneMap/WebODM) adds more features such as user authentication, map displays, 3D displays, a higher level API and the ability to orchestrate multiple processing nodes (run jobs in parallel). Processing nodes are simply servers running [NodeODM](https://github.com/OpenDroneMap/node-OpenDroneMap).
|
||||
- [WebODM](https://github.com/OpenDroneMap/WebODM) adds more features such as user authentication, map displays, 3D displays, a higher level API and the ability to orchestrate multiple processing nodes (run jobs in parallel). Processing nodes are simply servers running [NodeODM](https://github.com/OpenDroneMap/NodeODM).
|
||||
|
||||
![webodm](https://cloud.githubusercontent.com/assets/1951843/25567386/5aeec7aa-2dba-11e7-9169-aca97b70db79.png)
|
||||
|
||||
|
@ -281,7 +281,7 @@ WebODM is built with scalability and performance in mind. While the default setu
|
|||
![Architecture](https://user-images.githubusercontent.com/1951843/36916884-3a269a7a-1e23-11e8-997a-a57cd6ca7950.png)
|
||||
|
||||
A few things to note:
|
||||
* We use Celery workers to do background tasks such as resizing images and processing task results, but we use an ad-hoc scheduling mechanism to communicate with node-OpenDroneMap (which processes the orthophotos, 3D models, etc.). The choice to use two separate systems for task scheduling is due to the flexibility that an ad-hoc mechanism gives us for certain operations (capture task output, persistent data and ability to restart tasks mid-way, communication via REST calls, etc.).
|
||||
* We use Celery workers to do background tasks such as resizing images and processing task results, but we use an ad-hoc scheduling mechanism to communicate with NodeODM (which processes the orthophotos, 3D models, etc.). The choice to use two separate systems for task scheduling is due to the flexibility that an ad-hoc mechanism gives us for certain operations (capture task output, persistent data and ability to restart tasks mid-way, communication via REST calls, etc.).
|
||||
* If loaded on multiple machines, Celery workers should all share their `app/media` directory with the Django application (via network shares). You can manage workers via `./worker.sh`
|
||||
|
||||
|
||||
|
|
|
@ -30,7 +30,7 @@ class ProcessingNodeFilter(FilterSet):
|
|||
|
||||
class Meta:
|
||||
model = ProcessingNode
|
||||
fields = ['has_available_options', 'id', 'hostname', 'port', 'api_version', 'queue_count', ]
|
||||
fields = ['has_available_options', 'id', 'hostname', 'port', 'api_version', 'queue_count', 'max_images', ]
|
||||
|
||||
class ProcessingNodeViewSet(viewsets.ModelViewSet):
|
||||
"""
|
||||
|
|
|
@ -24,9 +24,16 @@ class TaskIDsSerializer(serializers.BaseSerializer):
|
|||
class TaskSerializer(serializers.ModelSerializer):
|
||||
project = serializers.PrimaryKeyRelatedField(queryset=models.Project.objects.all())
|
||||
processing_node = serializers.PrimaryKeyRelatedField(queryset=ProcessingNode.objects.all())
|
||||
processing_node_name = serializers.SerializerMethodField()
|
||||
images_count = serializers.SerializerMethodField()
|
||||
can_rerun_from = serializers.SerializerMethodField()
|
||||
|
||||
def get_processing_node_name(self, obj):
|
||||
if obj.processing_node is not None:
|
||||
return str(obj.processing_node)
|
||||
else:
|
||||
return None
|
||||
|
||||
def get_images_count(self, obj):
|
||||
return obj.imageupload_set.count()
|
||||
|
||||
|
@ -39,7 +46,7 @@ class TaskSerializer(serializers.ModelSerializer):
|
|||
|
||||
TODO: this could be improved by returning an empty array if a task was created
|
||||
and purged by the processing node (which would require knowing how long a task is being kept
|
||||
see https://github.com/OpenDroneMap/node-OpenDroneMap/issues/32
|
||||
see https://github.com/OpenDroneMap/NodeODM/issues/32
|
||||
:return: array of valid rerun-from parameters
|
||||
"""
|
||||
if obj.processing_node is not None:
|
||||
|
|
|
@ -0,0 +1,27 @@
|
|||
# Generated by Django 2.0.3 on 2018-12-05 16:44
|
||||
|
||||
from django.db import migrations, models
|
||||
|
||||
|
||||
class Migration(migrations.Migration):
|
||||
|
||||
dependencies = [
|
||||
('app', '0021_auto_20180726_1746'),
|
||||
]
|
||||
|
||||
operations = [
|
||||
migrations.RemoveField(
|
||||
model_name='task',
|
||||
name='ground_control_points',
|
||||
),
|
||||
migrations.AddField(
|
||||
model_name='task',
|
||||
name='resize_progress',
|
||||
field=models.FloatField(blank=True, default=0.0, help_text="Value between 0 and 1 indicating the resize progress of this task's images"),
|
||||
),
|
||||
migrations.AddField(
|
||||
model_name='task',
|
||||
name='upload_progress',
|
||||
field=models.FloatField(blank=True, default=0.0, help_text="Value between 0 and 1 indicating the upload progress of this task's files to the processing node"),
|
||||
),
|
||||
]
|
|
@ -0,0 +1,18 @@
|
|||
# Generated by Django 2.0.3 on 2018-12-07 18:53
|
||||
|
||||
from django.db import migrations, models
|
||||
|
||||
|
||||
class Migration(migrations.Migration):
|
||||
|
||||
dependencies = [
|
||||
('app', '0022_auto_20181205_1644'),
|
||||
]
|
||||
|
||||
operations = [
|
||||
migrations.AddField(
|
||||
model_name='task',
|
||||
name='running_progress',
|
||||
field=models.FloatField(blank=True, default=0.0, help_text="Value between 0 and 1 indicating the running progress (estimated) of this task"),
|
||||
),
|
||||
]
|
|
@ -2,6 +2,7 @@ import logging
|
|||
import os
|
||||
import shutil
|
||||
import zipfile
|
||||
import time
|
||||
import uuid as uuid_module
|
||||
|
||||
import json
|
||||
|
@ -69,7 +70,13 @@ def validate_task_options(value):
|
|||
|
||||
|
||||
|
||||
def resize_image(image_path, resize_to):
|
||||
def resize_image(image_path, resize_to, done=None):
|
||||
"""
|
||||
:param image_path: path to the image
|
||||
:param resize_to: target size to resize this image to (largest side)
|
||||
:param done: optional callback
|
||||
:return: path and resize ratio
|
||||
"""
|
||||
try:
|
||||
im = Image.open(image_path)
|
||||
path, ext = os.path.splitext(image_path)
|
||||
|
@ -105,9 +112,16 @@ def resize_image(image_path, resize_to):
|
|||
logger.info("Resized {} to {}x{}".format(image_path, resized_width, resized_height))
|
||||
except IOError as e:
|
||||
logger.warning("Cannot resize {}: {}.".format(image_path, str(e)))
|
||||
if done is not None:
|
||||
done()
|
||||
return None
|
||||
|
||||
return {'path': image_path, 'resize_ratio': ratio}
|
||||
retval = {'path': image_path, 'resize_ratio': ratio}
|
||||
|
||||
if done is not None:
|
||||
done(retval)
|
||||
|
||||
return retval
|
||||
|
||||
class Task(models.Model):
|
||||
ASSETS_MAP = {
|
||||
|
@ -142,6 +156,22 @@ class Task(models.Model):
|
|||
(pending_actions.RESIZE, 'RESIZE'),
|
||||
)
|
||||
|
||||
# Not an exact science
|
||||
TASK_OUTPUT_MILESTONES = {
|
||||
'Running ODM Load Dataset Cell': 0.01,
|
||||
'Running ODM Load Dataset Cell - Finished': 0.05,
|
||||
'opensfm/bin/opensfm match_features': 0.10,
|
||||
'opensfm/bin/opensfm reconstruct': 0.20,
|
||||
'opensfm/bin/opensfm export_visualsfm': 0.30,
|
||||
'Running ODM Meshing Cell': 0.60,
|
||||
'Running MVS Texturing Cell': 0.65,
|
||||
'Running ODM Georeferencing Cell': 0.70,
|
||||
'Running ODM DEM Cell': 0.80,
|
||||
'Running ODM Orthophoto Cell': 0.85,
|
||||
'Running ODM OrthoPhoto Cell - Finished': 0.90,
|
||||
'Compressing all.zip:': 0.95
|
||||
}
|
||||
|
||||
id = models.UUIDField(primary_key=True, default=uuid_module.uuid4, unique=True, serialize=False, editable=False)
|
||||
|
||||
uuid = models.CharField(max_length=255, db_index=True, default='', blank=True, help_text="Identifier of the task (as returned by OpenDroneMap's REST API)")
|
||||
|
@ -155,7 +185,6 @@ class Task(models.Model):
|
|||
options = fields.JSONField(default=dict(), blank=True, help_text="Options that are being used to process this task", validators=[validate_task_options])
|
||||
available_assets = fields.ArrayField(models.CharField(max_length=80), default=list(), blank=True, help_text="List of available assets to download")
|
||||
console_output = models.TextField(null=False, default="", blank=True, help_text="Console output of the OpenDroneMap's process")
|
||||
ground_control_points = models.FileField(null=True, blank=True, upload_to=gcp_directory_path, help_text="Optional Ground Control Points file to use for processing")
|
||||
|
||||
orthophoto_extent = GeometryField(null=True, blank=True, srid=4326, help_text="Extent of the orthophoto created by OpenDroneMap")
|
||||
dsm_extent = GeometryField(null=True, blank=True, srid=4326, help_text="Extent of the DSM created by OpenDroneMap")
|
||||
|
@ -168,6 +197,15 @@ class Task(models.Model):
|
|||
public = models.BooleanField(default=False, help_text="A flag indicating whether this task is available to the public")
|
||||
resize_to = models.IntegerField(default=-1, help_text="When set to a value different than -1, indicates that the images for this task have been / will be resized to the size specified here before processing.")
|
||||
|
||||
upload_progress = models.FloatField(default=0.0,
|
||||
help_text="Value between 0 and 1 indicating the upload progress of this task's files to the processing node",
|
||||
blank=True)
|
||||
resize_progress = models.FloatField(default=0.0,
|
||||
help_text="Value between 0 and 1 indicating the resize progress of this task's images",
|
||||
blank=True)
|
||||
running_progress = models.FloatField(default=0.0,
|
||||
help_text="Value between 0 and 1 indicating the running progress (estimated) of this task",
|
||||
blank=True)
|
||||
|
||||
def __init__(self, *args, **kwargs):
|
||||
super(Task, self).__init__(*args, **kwargs)
|
||||
|
@ -289,6 +327,7 @@ class Task(models.Model):
|
|||
try:
|
||||
if self.pending_action == pending_actions.RESIZE:
|
||||
resized_images = self.resize_images()
|
||||
self.refresh_from_db()
|
||||
self.resize_gcp(resized_images)
|
||||
self.pending_action = None
|
||||
self.save()
|
||||
|
@ -333,8 +372,17 @@ class Task(models.Model):
|
|||
|
||||
images = [image.path() for image in self.imageupload_set.all()]
|
||||
|
||||
# Track upload progress, but limit the number of DB updates
|
||||
# to every 2 seconds (and always record the 100% progress)
|
||||
last_update = 0
|
||||
def callback(progress):
|
||||
nonlocal last_update
|
||||
if time.time() - last_update >= 2 or (progress >= 1.0 - 1e-6 and progress <= 1.0 + 1e-6):
|
||||
Task.objects.filter(pk=self.id).update(upload_progress=progress)
|
||||
last_update = time.time()
|
||||
|
||||
# This takes a while
|
||||
uuid = self.processing_node.process_new_task(images, self.name, self.options)
|
||||
uuid = self.processing_node.process_new_task(images, self.name, self.options, callback)
|
||||
|
||||
# Refresh task object before committing change
|
||||
self.refresh_from_db()
|
||||
|
@ -400,12 +448,14 @@ class Task(models.Model):
|
|||
|
||||
# We also remove the "rerun-from" parameter if it's set
|
||||
self.options = list(filter(lambda d: d['name'] != 'rerun-from', self.options))
|
||||
self.upload_progress = 0
|
||||
|
||||
self.console_output = ""
|
||||
self.processing_time = -1
|
||||
self.status = None
|
||||
self.last_error = None
|
||||
self.pending_action = None
|
||||
self.running_progress = 0
|
||||
self.save()
|
||||
else:
|
||||
raise ProcessingError("Cannot restart a task that has no processing node")
|
||||
|
@ -439,7 +489,14 @@ class Task(models.Model):
|
|||
current_lines_count = len(self.console_output.split("\n"))
|
||||
console_output = self.processing_node.get_task_console_output(self.uuid, current_lines_count)
|
||||
if len(console_output) > 0:
|
||||
self.console_output += console_output + '\n'
|
||||
self.console_output += "\n".join(console_output) + '\n'
|
||||
|
||||
# Update running progress
|
||||
for line in console_output:
|
||||
for line_match, value in self.TASK_OUTPUT_MILESTONES.items():
|
||||
if line_match in line:
|
||||
self.running_progress = value
|
||||
break
|
||||
|
||||
if "errorMessage" in info["status"]:
|
||||
self.last_error = info["status"]["errorMessage"]
|
||||
|
@ -498,6 +555,7 @@ class Task(models.Model):
|
|||
logger.info("Populated extent field with {} for {}".format(raster_path, self))
|
||||
|
||||
self.update_available_assets_field()
|
||||
self.running_progress = 1.0
|
||||
self.save()
|
||||
|
||||
from app.plugins import signals as plugin_signals
|
||||
|
@ -620,12 +678,27 @@ class Task(models.Model):
|
|||
return []
|
||||
|
||||
images_path = self.find_all_files_matching(r'.*\.jpe?g$')
|
||||
total_images = len(images_path)
|
||||
resized_images_count = 0
|
||||
last_update = 0
|
||||
|
||||
def callback(retval=None):
|
||||
nonlocal last_update
|
||||
nonlocal resized_images_count
|
||||
nonlocal total_images
|
||||
|
||||
resized_images_count += 1
|
||||
if time.time() - last_update >= 2:
|
||||
Task.objects.filter(pk=self.id).update(resize_progress=(float(resized_images_count) / float(total_images)))
|
||||
last_update = time.time()
|
||||
|
||||
with ThreadPoolExecutor(max_workers=cpu_count()) as executor:
|
||||
resized_images = list(filter(lambda i: i is not None, executor.map(
|
||||
partial(resize_image, resize_to=self.resize_to),
|
||||
partial(resize_image, resize_to=self.resize_to, done=callback),
|
||||
images_path)))
|
||||
|
||||
Task.objects.filter(pk=self.id).update(resize_progress=1.0)
|
||||
|
||||
return resized_images
|
||||
|
||||
def resize_gcp(self, resized_images):
|
||||
|
|
|
@ -264,4 +264,8 @@ footer{
|
|||
.full-height{
|
||||
height: calc(100vh - 110px);
|
||||
padding-bottom: 12px;
|
||||
}
|
||||
|
||||
.floatfix{
|
||||
clear: both;
|
||||
}
|
|
@ -131,6 +131,14 @@ a, a:hover, a:focus{
|
|||
}
|
||||
}
|
||||
|
||||
.theme-color-button-danger{
|
||||
color: theme("button_danger");
|
||||
}
|
||||
|
||||
.theme-color-button-primary{
|
||||
color: theme("button_primary");
|
||||
}
|
||||
|
||||
/* Header background */
|
||||
#navbar-top{
|
||||
background-color: theme("header_background");
|
||||
|
@ -204,7 +212,6 @@ pre.prettyprint,
|
|||
background-color: theme("failed");
|
||||
}
|
||||
|
||||
|
||||
/* ModelView.jsx specific */
|
||||
.model-view #potree_sidebar_container {
|
||||
.dropdown-menu > li > a{
|
||||
|
|
|
@ -24,6 +24,8 @@ class Console extends React.Component {
|
|||
this.setRef = this.setRef.bind(this);
|
||||
this.handleMouseOver = this.handleMouseOver.bind(this);
|
||||
this.handleMouseOut = this.handleMouseOut.bind(this);
|
||||
this.downloadTxt = this.downloadTxt.bind(this);
|
||||
this.copyTxt = this.copyTxt.bind(this);
|
||||
}
|
||||
|
||||
componentDidMount(){
|
||||
|
@ -64,6 +66,7 @@ class Console extends React.Component {
|
|||
}
|
||||
|
||||
downloadTxt(filename="console.txt"){
|
||||
console.log(filename);
|
||||
function saveAs(uri, filename) {
|
||||
let link = document.createElement('a');
|
||||
if (typeof link.download === 'string') {
|
||||
|
@ -93,7 +96,7 @@ class Console extends React.Component {
|
|||
el.select();
|
||||
document.execCommand('copy');
|
||||
document.body.removeChild(el);
|
||||
console.log("Task output copied to clipboard");
|
||||
console.log("Output copied to clipboard");
|
||||
}
|
||||
|
||||
tearDownDynamicSource(){
|
||||
|
@ -140,22 +143,40 @@ class Console extends React.Component {
|
|||
}
|
||||
let i = 0;
|
||||
|
||||
return (
|
||||
<pre className={`console prettyprint
|
||||
${this.props.lang ? `lang-${this.props.lang}` : ""}
|
||||
${this.props.lines ? "linenums" : ""}`}
|
||||
style={{height: (this.props.height ? this.props.height : "auto")}}
|
||||
onMouseOver={this.handleMouseOver}
|
||||
onMouseOut={this.handleMouseOut}
|
||||
ref={this.setRef}
|
||||
>
|
||||
{this.state.lines.map(line => {
|
||||
if (this.props.lang) return (<div key={i++} dangerouslySetInnerHTML={prettyLine(line)}></div>);
|
||||
else return line + "\n";
|
||||
})}
|
||||
{"\n"}
|
||||
</pre>
|
||||
);
|
||||
let lines = this.state.lines;
|
||||
if (this.props.maximumLines && lines.length > this.props.maximumLines){
|
||||
lines = lines.slice(-this.props.maximumLines);
|
||||
lines.unshift(`... output truncated at ${this.props.maximumLines} lines ...`);
|
||||
}
|
||||
|
||||
const items = [
|
||||
<pre key="console" className={`console prettyprint
|
||||
${this.props.lang ? `lang-${this.props.lang}` : ""}
|
||||
${this.props.lines ? "linenums" : ""}
|
||||
${this.props.className || ""}`}
|
||||
style={{height: (this.props.height ? this.props.height : "auto")}}
|
||||
onMouseOver={this.handleMouseOver}
|
||||
onMouseOut={this.handleMouseOut}
|
||||
ref={this.setRef}
|
||||
>{lines.map(line => {
|
||||
if (this.props.lang) return (<div key={i++} dangerouslySetInnerHTML={prettyLine(line)}></div>);
|
||||
else return line + "\n";
|
||||
})}
|
||||
{"\n"}
|
||||
</pre>];
|
||||
|
||||
if (this.props.showConsoleButtons){
|
||||
items.push(<div key="buttons" className="console-buttons">
|
||||
<a href="javascript:void(0);" onClick={() => this.downloadTxt()} className="btn btn-sm btn-primary" title="Download To File">
|
||||
<i className="fa fa-download"></i>
|
||||
</a>
|
||||
<a href="javascript:void(0);" onClick={this.copyTxt} className="btn btn-sm btn-primary" title="Copy To Clipboard">
|
||||
<i className="fa fa-clipboard"></i>
|
||||
</a>
|
||||
</div>);
|
||||
}
|
||||
|
||||
return items;
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
@ -0,0 +1,16 @@
|
|||
const values = {};
|
||||
export default {
|
||||
getValue: function(className, property, element = 'div'){
|
||||
const k = className + '|' + property;
|
||||
if (values[k]) return values[k];
|
||||
else{
|
||||
let d = document.createElement(element);
|
||||
d.style.display = "none";
|
||||
d.className = className;
|
||||
document.body.appendChild(d);
|
||||
values[k] = getComputedStyle(d)[property];
|
||||
document.body.removeChild(d);
|
||||
return values[k];
|
||||
}
|
||||
}
|
||||
}
|
|
@ -0,0 +1,54 @@
|
|||
export default {
|
||||
get: function(){
|
||||
return [{
|
||||
action: "dataset",
|
||||
label: "Load Dataset",
|
||||
icon: "fa fa-database",
|
||||
beginsWith: "Running ODM Load Dataset Cell",
|
||||
endsWith: "Running ODM Load Dataset Cell - Finished"
|
||||
},
|
||||
{
|
||||
action: "opensfm",
|
||||
label: "Structure From Motion / MVS",
|
||||
icon: "fa fa-camera",
|
||||
beginsWith: "Running ODM OpenSfM Cell",
|
||||
endsWith: "Running ODM Meshing Cell"
|
||||
},
|
||||
{
|
||||
action: "odm_meshing",
|
||||
label: "Meshing",
|
||||
icon: "fa fa-cube",
|
||||
beginsWith: "Running ODM Meshing Cell",
|
||||
endsWith: "Running ODM Meshing Cell - Finished"
|
||||
},
|
||||
{
|
||||
action: "mvs_texturing",
|
||||
label: "Texturing",
|
||||
icon: "fa fa-connectdevelop",
|
||||
beginsWith: "Running MVS Texturing Cell",
|
||||
endsWith: "Running ODM Texturing Cell - Finished"
|
||||
},
|
||||
{
|
||||
action: "odm_georeferencing",
|
||||
label: "Georeferencing",
|
||||
icon: "fa fa-globe",
|
||||
beginsWith: "Running ODM Georeferencing Cell",
|
||||
endsWith: "Running ODM Georeferencing Cell - Finished"
|
||||
},
|
||||
{
|
||||
action: "odm_dem",
|
||||
label: "DEM",
|
||||
icon: "fa fa-area-chart",
|
||||
beginsWith: "Running ODM DEM Cell",
|
||||
endsWith: "Running ODM DEM Cell - Finished"
|
||||
},
|
||||
{
|
||||
action: "odm_orthophoto",
|
||||
label: "Orthophoto",
|
||||
icon: "fa fa-map-o",
|
||||
beginsWith: "Running ODM Orthophoto Cell",
|
||||
endsWith: "Running ODM OrthoPhoto Cell - Finished"
|
||||
}
|
||||
];
|
||||
}
|
||||
};
|
|
@ -45,7 +45,7 @@ class AssetDownloadButtons extends React.Component {
|
|||
{assetDownloads.map((asset, i) => {
|
||||
if (!asset.separator){
|
||||
return (<li key={i}>
|
||||
<a href="javascript:void(0);" onClick={this.downloadAsset(asset)}><i className={asset.icon}></i> {asset.label}</a>
|
||||
<a href="javascript:void(0);" onClick={this.downloadAsset(asset)}><i className={asset.icon + " fa-fw"}></i> {asset.label}</a>
|
||||
</li>);
|
||||
}else{
|
||||
return (<li key={i} className="divider"></li>);
|
||||
|
|
|
@ -0,0 +1,234 @@
|
|||
import React from 'react';
|
||||
import PropTypes from 'prop-types';
|
||||
import '../css/BasicTaskView.scss';
|
||||
import update from 'immutability-helper';
|
||||
import PipelineSteps from '../classes/PipelineSteps';
|
||||
import StatusCodes from '../classes/StatusCodes';
|
||||
import $ from 'jquery';
|
||||
|
||||
class BasicTaskView extends React.Component {
|
||||
static defaultProps = {};
|
||||
|
||||
static propTypes = {
|
||||
source: PropTypes.oneOfType([
|
||||
PropTypes.string.isRequired,
|
||||
PropTypes.func.isRequired
|
||||
]),
|
||||
taskStatus: PropTypes.number,
|
||||
onAddLines: PropTypes.func
|
||||
};
|
||||
|
||||
constructor(props){
|
||||
super();
|
||||
|
||||
this.state = {
|
||||
lines: [],
|
||||
currentRf: 0,
|
||||
rf: PipelineSteps.get(),
|
||||
loaded: false
|
||||
};
|
||||
|
||||
this.imageUpload = {
|
||||
action: "imageupload",
|
||||
label: "Image Resize / Upload",
|
||||
icon: "fa fa-image"
|
||||
};
|
||||
|
||||
// Add a post processing step
|
||||
this.state.rf.push({
|
||||
action: "postprocessing",
|
||||
label: "Post Processing",
|
||||
icon: "fa fa-file-archive-o",
|
||||
beginsWith: "Running ODM OrthoPhoto Cell - Finished",
|
||||
endsWith: null
|
||||
});
|
||||
|
||||
this.addLines = this.addLines.bind(this);
|
||||
this.setupDynamicSource = this.setupDynamicSource.bind(this);
|
||||
this.reset = this.reset.bind(this);
|
||||
this.tearDownDynamicSource = this.tearDownDynamicSource.bind(this);
|
||||
this.getRfEndStatus = this.getRfEndStatus.bind(this);
|
||||
this.getRfRunningStatus = this.getRfRunningStatus.bind(this);
|
||||
this.suffixFor = this.suffixFor.bind(this);
|
||||
this.updateRfState = this.updateRfState.bind(this);
|
||||
this.getInitialStatus = this.getInitialStatus.bind(this);
|
||||
}
|
||||
|
||||
componentDidMount(){
|
||||
this.reset();
|
||||
}
|
||||
|
||||
setupDynamicSource(){
|
||||
const updateFromSource = () => {
|
||||
let sourceUrl = typeof this.props.source === 'function' ?
|
||||
this.props.source(this.state.lines.length) :
|
||||
this.props.source;
|
||||
|
||||
// Fetch
|
||||
this.sourceRequest = $.get(sourceUrl, text => {
|
||||
if (text !== ""){
|
||||
let lines = text.split("\n");
|
||||
this.addLines(lines);
|
||||
}
|
||||
})
|
||||
.always((_, textStatus) => {
|
||||
if (textStatus !== "abort" && this.props.refreshInterval !== undefined){
|
||||
this.sourceTimeout = setTimeout(updateFromSource, this.props.refreshInterval);
|
||||
}
|
||||
if (!this.state.loaded) this.setState({loaded: true});
|
||||
});
|
||||
};
|
||||
|
||||
updateFromSource();
|
||||
}
|
||||
|
||||
getRfEndStatus(){
|
||||
return this.props.taskStatus === StatusCodes.COMPLETED ?
|
||||
'completed' :
|
||||
'errored';
|
||||
}
|
||||
|
||||
getRfRunningStatus(){
|
||||
return [StatusCodes.RUNNING, StatusCodes.QUEUED].indexOf(this.props.taskStatus) !== -1 ?
|
||||
'running' :
|
||||
'errored';
|
||||
}
|
||||
|
||||
getInitialStatus(){
|
||||
if ([null, StatusCodes.QUEUED, StatusCodes.RUNNING].indexOf(this.props.taskStatus) !== -1){
|
||||
return 'queued';
|
||||
}else{
|
||||
return this.getRfEndStatus();
|
||||
}
|
||||
}
|
||||
|
||||
reset(){
|
||||
this.state.rf.forEach(p => {
|
||||
p.state = this.getInitialStatus();
|
||||
});
|
||||
|
||||
if ([StatusCodes.RUNNING].indexOf(this.props.taskStatus) !== -1){
|
||||
this.state.rf[0].state = 'running';
|
||||
}
|
||||
|
||||
this.tearDownDynamicSource();
|
||||
this.setState({lines: [], currentRf: 0, loaded: false, rf: this.state.rf});
|
||||
this.setupDynamicSource();
|
||||
}
|
||||
|
||||
tearDownDynamicSource(){
|
||||
if (this.sourceTimeout) clearTimeout(this.sourceTimeout);
|
||||
if (this.sourceRequest) this.sourceRequest.abort();
|
||||
}
|
||||
|
||||
componentDidUpdate(prevProps){
|
||||
|
||||
let taskFailed;
|
||||
let taskCompleted;
|
||||
let taskRestarted;
|
||||
|
||||
if (prevProps.taskStatus !== this.props.taskStatus){
|
||||
taskFailed = [StatusCodes.FAILED, StatusCodes.CANCELED].indexOf(this.props.taskStatus) !== -1;
|
||||
taskCompleted = this.props.taskStatus === StatusCodes.COMPLETED;
|
||||
taskRestarted = this.props.taskStatus === null;
|
||||
}
|
||||
|
||||
this.updateRfState(taskFailed, taskCompleted, taskRestarted);
|
||||
}
|
||||
|
||||
componentWillUnmount(){
|
||||
this.tearDownDynamicSource();
|
||||
}
|
||||
|
||||
addLines(lines){
|
||||
if (!Array.isArray(lines)) lines = [lines];
|
||||
|
||||
let currentRf = this.state.currentRf;
|
||||
|
||||
const updateRf = (rfIndex, line) => {
|
||||
const current = this.state.rf[rfIndex];
|
||||
if (!current) return;
|
||||
|
||||
if (current.beginsWith && line.endsWith(current.beginsWith)){
|
||||
current.state = this.getRfRunningStatus();
|
||||
|
||||
// Set previous as done
|
||||
if (this.state.rf[rfIndex - 1]){
|
||||
this.state.rf[rfIndex - 1].state = 'completed';
|
||||
}
|
||||
}else if (current.endsWith && line.endsWith(current.endsWith)){
|
||||
current.state = this.getRfEndStatus();
|
||||
|
||||
// Check next
|
||||
updateRf(rfIndex + 1, line);
|
||||
|
||||
currentRf = rfIndex + 1;
|
||||
}
|
||||
};
|
||||
|
||||
lines.forEach(line => {
|
||||
updateRf(currentRf, line);
|
||||
});
|
||||
|
||||
this.setState(update(this.state, {
|
||||
lines: {$push: lines}
|
||||
}));
|
||||
this.setState({
|
||||
rf: this.state.rf,
|
||||
currentRf
|
||||
});
|
||||
this.updateRfState();
|
||||
|
||||
if (this.props.onAddLines) this.props.onAddLines(lines);
|
||||
}
|
||||
|
||||
updateRfState(taskFailed, taskCompleted, taskRestarted){
|
||||
// If the task has just failed, update all items that were either running or in queued state
|
||||
if (taskFailed){
|
||||
this.state.rf.forEach(p => {
|
||||
if (p.state === 'queued' || p.state === 'running') p.state = this.getInitialStatus();
|
||||
});
|
||||
}
|
||||
|
||||
// If completed, all steps must have completed
|
||||
if (taskCompleted){
|
||||
this.state.rf.forEach(p => p.state = 'completed');
|
||||
}
|
||||
|
||||
if (taskRestarted){
|
||||
this.state.rf.forEach(p => p.state = 'queued');
|
||||
}
|
||||
}
|
||||
|
||||
suffixFor(state){
|
||||
if (state === 'running'){
|
||||
return (<span>...</span>);
|
||||
}else if (state === 'completed'){
|
||||
return (<i className="fa fa-check"></i>);
|
||||
}else if (state === 'errored'){
|
||||
return (<i className="fa fa-times"></i>);
|
||||
}
|
||||
}
|
||||
|
||||
render() {
|
||||
const { rf, loaded } = this.state;
|
||||
const imageUploadState = this.props.taskStatus === null
|
||||
? 'running'
|
||||
: 'completed';
|
||||
|
||||
return (<div className={"basic-task-view " + (loaded ? 'loaded' : '')}>
|
||||
<div className={imageUploadState + " processing-step"}>
|
||||
<i className={this.imageUpload.icon + " fa-fw"}></i> {this.imageUpload.label} {this.suffixFor(imageUploadState)}
|
||||
</div>
|
||||
{rf.map(p => {
|
||||
const state = loaded ? p.state : 'queued';
|
||||
|
||||
return (<div key={p.action} className={state + " processing-step"}>
|
||||
<i className={p.icon + " fa-fw"}></i> {p.label} {this.suffixFor(state)}
|
||||
</div>);
|
||||
})}
|
||||
</div>);
|
||||
}
|
||||
}
|
||||
|
||||
export default BasicTaskView;
|
|
@ -79,7 +79,8 @@ class ProjectListItem extends React.Component {
|
|||
progress: 0,
|
||||
totalCount: 0,
|
||||
totalBytes: 0,
|
||||
totalBytesSent: 0
|
||||
totalBytesSent: 0,
|
||||
lastUpdated: 0
|
||||
};
|
||||
}
|
||||
|
||||
|
@ -152,9 +153,18 @@ class ProjectListItem extends React.Component {
|
|||
});
|
||||
|
||||
this.dz.on("totaluploadprogress", (progress, totalBytes, totalBytesSent) => {
|
||||
this.setUploadState({
|
||||
progress, totalBytes, totalBytesSent
|
||||
});
|
||||
// Limit updates since this gets called a lot
|
||||
let now = (new Date()).getTime();
|
||||
|
||||
// Progress 100 is sent multiple times at the end
|
||||
// this makes it so that we update the state only once.
|
||||
if (progress === 100) now = now + 9999999999;
|
||||
|
||||
if (this.state.upload.lastUpdated + 500 < now){
|
||||
this.setUploadState({
|
||||
progress, totalBytes, totalBytesSent, lastUpdated: now
|
||||
});
|
||||
}
|
||||
})
|
||||
.on("addedfiles", files => {
|
||||
this.setUploadState({
|
||||
|
@ -164,7 +174,7 @@ class ProjectListItem extends React.Component {
|
|||
})
|
||||
.on("transformcompleted", (file, total) => {
|
||||
if (this.dz._resizeMap) this.dz._resizeMap[file.name] = this.dz._taskInfo.resizeSize / Math.max(file.width, file.height);
|
||||
this.setUploadState({resizedImages: total});
|
||||
if (this.dz.options.resizeWidth) this.setUploadState({resizedImages: total});
|
||||
})
|
||||
.on("transformstart", (files) => {
|
||||
if (this.dz.options.resizeWidth){
|
||||
|
|
|
@ -9,6 +9,9 @@ import AssetDownloadButtons from './AssetDownloadButtons';
|
|||
import HistoryNav from '../classes/HistoryNav';
|
||||
import PropTypes from 'prop-types';
|
||||
import TaskPluginActionButtons from './TaskPluginActionButtons';
|
||||
import PipelineSteps from '../classes/PipelineSteps';
|
||||
import BasicTaskView from './BasicTaskView';
|
||||
import Css from '../classes/Css';
|
||||
|
||||
class TaskListItem extends React.Component {
|
||||
static propTypes = {
|
||||
|
@ -32,7 +35,8 @@ class TaskListItem extends React.Component {
|
|||
editing: false,
|
||||
memoryError: false,
|
||||
friendlyTaskError: "",
|
||||
pluginActionButtons: []
|
||||
pluginActionButtons: [],
|
||||
view: "basic"
|
||||
}
|
||||
|
||||
for (let k in props.data){
|
||||
|
@ -44,9 +48,12 @@ class TaskListItem extends React.Component {
|
|||
this.stopEditing = this.stopEditing.bind(this);
|
||||
this.startEditing = this.startEditing.bind(this);
|
||||
this.checkForCommonErrors = this.checkForCommonErrors.bind(this);
|
||||
this.downloadTaskOutput = this.downloadTaskOutput.bind(this);
|
||||
this.copyTaskOutput = this.copyTaskOutput.bind(this);
|
||||
this.handleEditTaskSave = this.handleEditTaskSave.bind(this);
|
||||
this.setView = this.setView.bind(this);
|
||||
|
||||
// Retrieve CSS values for status bar colors
|
||||
this.backgroundSuccessColor = Css.getValue('theme-background-success', 'backgroundColor');
|
||||
this.backgroundFailedColor = Css.getValue('theme-background-failed', 'backgroundColor');
|
||||
}
|
||||
|
||||
shouldRefresh(){
|
||||
|
@ -69,6 +76,12 @@ class TaskListItem extends React.Component {
|
|||
}
|
||||
}
|
||||
|
||||
setView(type){
|
||||
return () => {
|
||||
this.setState({view: type});
|
||||
}
|
||||
}
|
||||
|
||||
unloadTimer(){
|
||||
if (this.processingTimeInterval){
|
||||
clearInterval(this.processingTimeInterval);
|
||||
|
@ -96,6 +109,7 @@ class TaskListItem extends React.Component {
|
|||
if (oldStatus !== this.state.task.status){
|
||||
if (this.state.task.status === statusCodes.RUNNING){
|
||||
if (this.console) this.console.clear();
|
||||
if (this.basicView) this.basicView.reset();
|
||||
this.loadTimer(this.state.task.processing_time);
|
||||
}else{
|
||||
this.setState({time: this.state.task.processing_time});
|
||||
|
@ -205,14 +219,6 @@ class TaskListItem extends React.Component {
|
|||
};
|
||||
}
|
||||
|
||||
downloadTaskOutput(){
|
||||
this.console.downloadTxt("task_output.txt");
|
||||
}
|
||||
|
||||
copyTaskOutput(){
|
||||
this.console.copyTxt();
|
||||
}
|
||||
|
||||
optionsToList(options){
|
||||
if (!Array.isArray(options)) return "";
|
||||
else if (options.length === 0) return "Default";
|
||||
|
@ -269,35 +275,13 @@ class TaskListItem extends React.Component {
|
|||
const { task } = this.state;
|
||||
|
||||
// Map rerun-from parameters to display items
|
||||
const rfMap = {
|
||||
"odm_meshing": {
|
||||
label: "From Meshing",
|
||||
icon: "fa fa-cube"
|
||||
},
|
||||
|
||||
"mvs_texturing": {
|
||||
label: "From Texturing",
|
||||
icon: "fa fa-connectdevelop"
|
||||
},
|
||||
|
||||
"odm_georeferencing": {
|
||||
label: "From Georeferencing",
|
||||
icon: "fa fa-globe"
|
||||
},
|
||||
|
||||
"odm_dem": {
|
||||
label: "From DEM",
|
||||
icon: "fa fa-area-chart"
|
||||
},
|
||||
|
||||
"odm_orthophoto": {
|
||||
label: "From Orthophoto",
|
||||
icon: "fa fa-map-o"
|
||||
}
|
||||
};
|
||||
// (remove the first item so that 'dataset' is not displayed)
|
||||
const rfMap = {};
|
||||
PipelineSteps.get().slice(1).forEach(rf => rfMap[rf.action] = rf);
|
||||
|
||||
// Create onClick handlers
|
||||
for (let rfParam in rfMap){
|
||||
rfMap[rfParam].label = "From " + rfMap[rfParam].label;
|
||||
rfMap[rfParam].onClick = this.genRestartAction(rfParam);
|
||||
}
|
||||
|
||||
|
@ -372,7 +356,7 @@ class TaskListItem extends React.Component {
|
|||
const name = task.name !== null ? task.name : `Task #${task.id}`;
|
||||
|
||||
let status = statusCodes.description(task.status);
|
||||
if (status === "") status = "Uploading images";
|
||||
if (status === "") status = "Uploading images to processing node";
|
||||
|
||||
if (!task.processing_node) status = "Waiting for a node...";
|
||||
if (task.pending_action !== null) status = pendingActions.description(task.pending_action);
|
||||
|
@ -468,24 +452,23 @@ class TaskListItem extends React.Component {
|
|||
data-toggle="dropdown"><span className="caret"></span></button>,
|
||||
<ul key="dropdown-menu" className="dropdown-menu">
|
||||
{subItems.map(subItem => <li key={subItem.label}>
|
||||
<a href="javascript:void(0);" onClick={subItem.onClick}><i className={subItem.icon}></i>{subItem.label}</a>
|
||||
<a href="javascript:void(0);" onClick={subItem.onClick}><i className={subItem.icon + ' fa-fw '}></i>{subItem.label}</a>
|
||||
</li>)}
|
||||
</ul>]}
|
||||
</div>);
|
||||
})}
|
||||
</div>);
|
||||
|
||||
|
||||
expanded = (
|
||||
<div className="expanded-panel">
|
||||
<div className="row">
|
||||
<div className="col-md-4 no-padding">
|
||||
<div className="col-md-3 no-padding">
|
||||
<div className="labels">
|
||||
<strong>Created on: </strong> {(new Date(task.created_at)).toLocaleString()}<br/>
|
||||
</div>
|
||||
{status ? <div className="labels">
|
||||
<strong>Status: </strong> {status}<br/>
|
||||
</div>
|
||||
: ""}
|
||||
<div className="labels">
|
||||
<strong>Processing Node: </strong> {task.processing_node_name || "-"} ({task.auto_processing_node ? "auto" : "manual"})<br/>
|
||||
</div>
|
||||
{Array.isArray(task.options) ?
|
||||
<div className="labels">
|
||||
<strong>Options: </strong> {this.optionsToList(task.options)}<br/>
|
||||
|
@ -497,27 +480,39 @@ class TaskListItem extends React.Component {
|
|||
<div className="task-warning"><i className="fa fa-warning"></i> <span>An orthophoto could not be generated. To generate one, make sure GPS information is embedded in the EXIF tags of your images, or use a Ground Control Points (GCP) file.</span></div> : ""}
|
||||
|
||||
</div>
|
||||
<div className="col-md-8">
|
||||
<Console
|
||||
source={this.consoleOutputUrl}
|
||||
refreshInterval={this.shouldRefresh() ? 3000 : undefined}
|
||||
autoscroll={true}
|
||||
height={200}
|
||||
ref={domNode => this.console = domNode}
|
||||
onAddLines={this.checkForCommonErrors}
|
||||
/>
|
||||
|
||||
<div className="console-buttons">
|
||||
<a href="javascript:void(0);" onClick={this.downloadTaskOutput} className="btn btn-sm btn-primary" title="Download Task Output">
|
||||
<i className="fa fa-download"></i>
|
||||
</a>
|
||||
<a href="javascript:void(0);" onClick={this.copyTaskOutput} className="btn btn-sm btn-primary" title="Copy Task Output">
|
||||
<i className="fa fa-clipboard"></i>
|
||||
</a>
|
||||
<div className="col-md-9">
|
||||
<div className="switch-view text-right pull-right">
|
||||
<i className="fa fa-list-ul"></i> <a href="javascript:void(0);" onClick={this.setView("basic")}
|
||||
className={this.state.view === 'basic' ? "selected" : ""}>Simple</a>
|
||||
|
|
||||
<i className="fa fa-desktop"></i> <a href="javascript:void(0);" onClick={this.setView("console")}
|
||||
className={this.state.view === 'console' ? "selected" : ""}>Console</a>
|
||||
</div>
|
||||
|
||||
{this.state.view === 'console' ?
|
||||
<Console
|
||||
className="floatfix"
|
||||
source={this.consoleOutputUrl}
|
||||
refreshInterval={this.shouldRefresh() ? 3000 : undefined}
|
||||
autoscroll={true}
|
||||
height={200}
|
||||
ref={domNode => this.console = domNode}
|
||||
onAddLines={this.checkForCommonErrors}
|
||||
showConsoleButtons={true}
|
||||
maximumLines={500}
|
||||
/> : ""}
|
||||
|
||||
{this.state.view === 'basic' ?
|
||||
<BasicTaskView
|
||||
source={this.consoleOutputUrl}
|
||||
ref={domNode => this.basicView = domNode}
|
||||
refreshInterval={this.shouldRefresh() ? 3000 : undefined}
|
||||
onAddLines={this.checkForCommonErrors}
|
||||
taskStatus={task.status}
|
||||
/> : ""}
|
||||
|
||||
{showMemoryErrorWarning ?
|
||||
<div className="task-warning"><i className="fa fa-support"></i> <span>It looks like your processing node ran out of memory. If you are using docker, make sure that your docker environment has <a href={memoryErrorLink} target="_blank">enough RAM allocated</a>. Alternatively, make sure you have enough physical RAM, reduce the number of images, make your images smaller, or reduce the max-concurrency parameter from the task's <a href="javascript:void(0);" onClick={this.startEditing}>options</a>.</span></div> : ""}
|
||||
<div className="task-warning"><i className="fa fa-support"></i> <span>It looks like your processing node ran out of memory. If you are using docker, make sure that your docker environment has <a href={memoryErrorLink} target="_blank">enough RAM allocated</a>. Alternatively, make sure you have enough physical RAM, reduce the number of images, make your images smaller, or reduce the max-concurrency parameter from the task's <a href="javascript:void(0);" onClick={this.startEditing}>options</a>. You can also try to use a <a href="https://www.opendronemap.org/webodm/lightning/" target="_blank">cloud processing node</a>.</span></div> : ""}
|
||||
|
||||
{showTaskWarning ?
|
||||
<div className="task-warning"><i className="fa fa-support"></i> <span dangerouslySetInnerHTML={{__html: this.state.friendlyTaskError}} /></div> : ""}
|
||||
|
@ -529,7 +524,7 @@ class TaskListItem extends React.Component {
|
|||
<li>Increase the <b>min-num-features</b> option, especially if your images have lots of vegetation</li>
|
||||
</ul>
|
||||
Still not working? Upload your images somewhere like <a href="https://www.dropbox.com/" target="_blank">Dropbox</a> or <a href="https://drive.google.com/drive/u/0/" target="_blank">Google Drive</a> and <a href="http://community.opendronemap.org/c/webodm" target="_blank">open a topic</a> on our community forum, making
|
||||
sure to include a <a href="javascript:void(0);" onClick={this.downloadTaskOutput}>copy of your task's output</a> (the one you see above <i className="fa fa-arrow-up"></i>, click to <a href="javascript:void(0);" onClick={this.downloadTaskOutput}>download</a> it). Our awesome contributors will try to help you! <i className="fa fa-smile-o"></i>
|
||||
sure to include a <a href="javascript:void(0);" onClick={this.setView("console")}>copy of your task's output</a>. Our awesome contributors will try to help you! <i className="fa fa-smile-o"></i>
|
||||
</div>
|
||||
</div>
|
||||
: ""}
|
||||
|
@ -557,8 +552,15 @@ class TaskListItem extends React.Component {
|
|||
}
|
||||
}
|
||||
|
||||
const getStatusLabel = (text, classes = "") => {
|
||||
return (<div className={"status-label " + classes} title={text}>{text}</div>);
|
||||
// @param type {String} one of: ['neutral', 'done', 'error']
|
||||
const getStatusLabel = (text, type = 'neutral', progress = 100) => {
|
||||
let color = 'rgba(255, 255, 255, 0.0)';
|
||||
if (type === 'done') color = this.backgroundSuccessColor;
|
||||
else if (type === 'error') color = this.backgroundFailedColor;
|
||||
return (<div
|
||||
className={"status-label theme-border-primary " + type}
|
||||
style={{background: `linear-gradient(90deg, ${color} ${progress}%, rgba(255, 255, 255, 0) ${progress}%)`}}
|
||||
title={text}>{text}</div>);
|
||||
}
|
||||
|
||||
let statusLabel = "";
|
||||
|
@ -566,13 +568,28 @@ class TaskListItem extends React.Component {
|
|||
let showEditLink = false;
|
||||
|
||||
if (task.last_error){
|
||||
statusLabel = getStatusLabel(task.last_error, "error");
|
||||
statusLabel = getStatusLabel(task.last_error, 'error');
|
||||
}else if (!task.processing_node){
|
||||
statusLabel = getStatusLabel("Set a processing node");
|
||||
statusIcon = "fa fa-hourglass-3";
|
||||
showEditLink = true;
|
||||
}else{
|
||||
statusLabel = getStatusLabel(status, task.status == 40 ? "done" : "");
|
||||
let progress = 100;
|
||||
let type = 'done';
|
||||
|
||||
if (task.pending_action === pendingActions.RESIZE){
|
||||
progress = task.resize_progress * 100;
|
||||
}else if (task.status === null){
|
||||
progress = task.upload_progress * 100;
|
||||
}else if (task.status === statusCodes.RUNNING){
|
||||
progress = task.running_progress * 100;
|
||||
}else if (task.status === statusCodes.FAILED){
|
||||
type = 'error';
|
||||
}else if (task.status !== statusCodes.COMPLETED){
|
||||
type = 'neutral';
|
||||
}
|
||||
|
||||
statusLabel = getStatusLabel(status, type, progress);
|
||||
}
|
||||
|
||||
return (
|
||||
|
|
|
@ -0,0 +1,10 @@
|
|||
import React from 'react';
|
||||
import { mount } from 'enzyme';
|
||||
import BasicTaskView from '../BasicTaskView';
|
||||
|
||||
describe('<BasicTaskView />', () => {
|
||||
it('renders without exploding', () => {
|
||||
const wrapper = mount(<BasicTaskView source="http://localhost/output" taskStatus={40} />);
|
||||
expect(wrapper.exists()).toBe(true);
|
||||
})
|
||||
});
|
|
@ -0,0 +1,38 @@
|
|||
.basic-task-view{
|
||||
margin-bottom: 8px;
|
||||
opacity: 0.7;
|
||||
&.loaded{
|
||||
opacity: 1;
|
||||
}
|
||||
|
||||
.processing-step{
|
||||
opacity: 0.7;
|
||||
|
||||
&.completed{
|
||||
opacity: 1;
|
||||
/* font-weight: bold; */
|
||||
}
|
||||
&.running{
|
||||
opacity: 1;
|
||||
animation: pulse ease-in-out 1.4833s infinite;
|
||||
}
|
||||
&.queued{
|
||||
|
||||
}
|
||||
&.errored{
|
||||
opacity: 1;
|
||||
}
|
||||
}
|
||||
|
||||
@keyframes pulse {
|
||||
0% {
|
||||
opacity: 0.7;
|
||||
}
|
||||
50% {
|
||||
opacity: 1;
|
||||
}
|
||||
100% {
|
||||
opacity: 0.7;
|
||||
}
|
||||
}
|
||||
}
|
|
@ -0,0 +1,9 @@
|
|||
.console-buttons{
|
||||
margin-left: 16px;
|
||||
margin-bottom: 16px;
|
||||
float: right;
|
||||
text-align: right;
|
||||
a{
|
||||
margin-left: 4px;
|
||||
}
|
||||
}
|
|
@ -41,6 +41,8 @@
|
|||
padding: 4px;
|
||||
width: 100%;
|
||||
font-size: 90%;
|
||||
border-style: solid;
|
||||
border-width: 1px;
|
||||
}
|
||||
|
||||
.clickable:hover{
|
||||
|
@ -97,13 +99,25 @@
|
|||
display: inline;
|
||||
}
|
||||
|
||||
.console-buttons{
|
||||
margin-left: 16px;
|
||||
margin-bottom: 16px;
|
||||
float: right;
|
||||
text-align: right;
|
||||
.switch-view{
|
||||
margin-bottom: 8px;
|
||||
font-size: 90%;
|
||||
position: relative;
|
||||
z-index: 99;
|
||||
|
||||
a{
|
||||
margin-left: 4px;
|
||||
margin-right: 8px;
|
||||
&.selected{
|
||||
font-weight: bold;
|
||||
}
|
||||
&:last-child{
|
||||
margin-right: 0;
|
||||
}
|
||||
}
|
||||
|
||||
i{
|
||||
margin-left: 8px;
|
||||
}
|
||||
}
|
||||
|
||||
}
|
|
@ -14,7 +14,7 @@
|
|||
{% if no_processingnodes %}
|
||||
<h3>{% trans 'Welcome! ☺' %}</h3>
|
||||
{% trans 'Add a Processing Node' as add_processing_node %}
|
||||
{% with nodeodm_link='<a href="https://github.com/pierotofy/node-OpenDroneMap" target="_blank">node-OpenDroneMap</a>' api_link='<a href="https://github.com/pierotofy/node-OpenDroneMap/blob/master/docs/index.adoc" target="_blank">API</a>' %}
|
||||
{% with nodeodm_link='<a href="https://github.com/OpenDroneMap/NodeODM" target="_blank">NodeODM</a>' api_link='<a href="https://github.com/OpenDroneMap/NodeODM/blob/master/docs/index.adoc" target="_blank">API</a>' %}
|
||||
<p>
|
||||
{% blocktrans %}
|
||||
To get started, "{{ add_processing_node }}". A processing node is a computer running an instance of {{ nodeodm_link }} or some other software that implements this {{ api_link }}.
|
||||
|
|
|
@ -254,7 +254,7 @@
|
|||
<ul class="nav nav-second-level">
|
||||
{% for node in nodes %}
|
||||
<li>
|
||||
<a href="{% url 'processing_node' node.id %}"><span class="fa fa-laptop"></span> {{node}}</a>
|
||||
<a href="{% url 'processing_node' node.id %}"><span class="fa fa-laptop {% if node.is_online %}theme-color-button-primary{% else %}theme-color-button-danger{% endif %}"></span> {{node}}</a>
|
||||
</li>
|
||||
{% endfor %}
|
||||
|
||||
|
|
|
@ -23,6 +23,10 @@
|
|||
<td>{% trans "Queue Count" %}</td>
|
||||
<td>{{ processing_node.queue_count }}</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>{% trans "Max Images Limit" %}</td>
|
||||
<td>{{ processing_node.max_images }}</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>{% trans "Last Refreshed" %}</td>
|
||||
<td>{{ processing_node.last_refreshed|timesince }} {% trans 'ago' %} ({{ processing_node.last_refreshed|localtime }})</td> <!-- TODO: timezone? -->
|
||||
|
|
|
@ -350,6 +350,9 @@ class TestApi(BootTestCase):
|
|||
# Should be set to false
|
||||
self.assertFalse(res.data['online'])
|
||||
|
||||
# Verify max images field
|
||||
self.assertTrue("max_images" in res.data)
|
||||
|
||||
# Cannot delete a processing node as normal user
|
||||
res = client.delete('/api/processingnodes/{}/'.format(pnode.id))
|
||||
self.assertTrue(res.status_code, status.HTTP_403_FORBIDDEN)
|
||||
|
|
|
@ -127,7 +127,6 @@ class TestApiTask(BootTransactionTestCase):
|
|||
with Image.open(multiple_param_task.task_path("tiny_drone_image.jpg")) as im:
|
||||
self.assertTrue(im.size == img1.size)
|
||||
|
||||
|
||||
# Normal case with images[], GCP, name and processing node parameter and resize_to option
|
||||
gcp = open("app/fixtures/gcp.txt", 'r')
|
||||
res = client.post("/api/projects/{}/tasks/".format(project.id), {
|
||||
|
@ -161,6 +160,13 @@ class TestApiTask(BootTransactionTestCase):
|
|||
self.assertTrue(float(px) == 8.0) # Didn't change
|
||||
self.assertTrue(float(py) == 8.0) # Didn't change
|
||||
|
||||
# Resize progress is 100%
|
||||
resized_task.refresh_from_db()
|
||||
self.assertEqual(resized_task.resize_progress, 1.0)
|
||||
|
||||
# Upload progress is 100%
|
||||
self.assertEqual(resized_task.upload_progress, 1.0)
|
||||
|
||||
# Case with malformed GCP file option
|
||||
with open("app/fixtures/gcp_malformed.txt", 'r') as malformed_gcp:
|
||||
res = client.post("/api/projects/{}/tasks/".format(project.id), {
|
||||
|
@ -181,7 +187,6 @@ class TestApiTask(BootTransactionTestCase):
|
|||
image1.seek(0)
|
||||
image2.seek(0)
|
||||
|
||||
|
||||
# Cannot create a task with images[], name, but invalid processing node parameter
|
||||
res = client.post("/api/projects/{}/tasks/".format(project.id), {
|
||||
'images': [image1, image2],
|
||||
|
@ -206,12 +211,18 @@ class TestApiTask(BootTransactionTestCase):
|
|||
self.assertTrue('id' in res.data)
|
||||
self.assertTrue(str(task.id) == res.data['id'])
|
||||
|
||||
# Progress is at 0%
|
||||
self.assertEqual(task.running_progress, 0.0)
|
||||
|
||||
# Two images should have been uploaded
|
||||
self.assertTrue(ImageUpload.objects.filter(task=task).count() == 2)
|
||||
|
||||
# Can_rerun_from should be an empty list
|
||||
self.assertTrue(len(res.data['can_rerun_from']) == 0)
|
||||
|
||||
# processing_node_name should be null
|
||||
self.assertTrue(res.data['processing_node_name'] is None)
|
||||
|
||||
# No processing node is set
|
||||
self.assertTrue(task.processing_node is None)
|
||||
|
||||
|
@ -262,10 +273,33 @@ class TestApiTask(BootTransactionTestCase):
|
|||
# (during tests this is sync)
|
||||
|
||||
# Processing should have started and a UUID is assigned
|
||||
# Calling process pending tasks should finish the process
|
||||
# and invoke the plugins completed signal
|
||||
task.refresh_from_db()
|
||||
self.assertTrue(task.status in [status_codes.RUNNING, status_codes.COMPLETED]) # Sometimes the task finishes and we can't test for RUNNING state
|
||||
self.assertTrue(task.status in [status_codes.RUNNING, status_codes.COMPLETED]) # Sometimes this finishes before we get here
|
||||
self.assertTrue(len(task.uuid) > 0)
|
||||
|
||||
with catch_signal(task_completed) as handler:
|
||||
retry_count = 0
|
||||
while task.status != status_codes.COMPLETED:
|
||||
worker.tasks.process_pending_tasks()
|
||||
time.sleep(DELAY)
|
||||
task.refresh_from_db()
|
||||
retry_count += 1
|
||||
if retry_count > 10:
|
||||
break
|
||||
|
||||
self.assertEqual(task.status, status_codes.COMPLETED)
|
||||
|
||||
# Progress is 100%
|
||||
self.assertTrue(task.running_progress == 1.0)
|
||||
|
||||
handler.assert_any_call(
|
||||
sender=Task,
|
||||
task_id=task.id,
|
||||
signal=task_completed,
|
||||
)
|
||||
|
||||
# Processing node should have a "rerun_from" option
|
||||
pnode_rerun_from_opts = list(filter(lambda d: 'name' in d and d['name'] == 'rerun-from', pnode.available_options))[0]
|
||||
self.assertTrue(len(pnode_rerun_from_opts['domain']) > 0)
|
||||
|
@ -277,20 +311,8 @@ class TestApiTask(BootTransactionTestCase):
|
|||
self.assertTrue(res.status_code == status.HTTP_200_OK)
|
||||
self.assertTrue(pnode_rerun_from_opts['domain'] == res.data['can_rerun_from'])
|
||||
|
||||
time.sleep(DELAY)
|
||||
|
||||
# Calling process pending tasks should finish the process
|
||||
# and invoke the plugins completed signal
|
||||
with catch_signal(task_completed) as handler:
|
||||
worker.tasks.process_pending_tasks()
|
||||
task.refresh_from_db()
|
||||
self.assertTrue(task.status == status_codes.COMPLETED)
|
||||
|
||||
handler.assert_called_with(
|
||||
sender=Task,
|
||||
task_id=task.id,
|
||||
signal=task_completed,
|
||||
)
|
||||
# processing_node_name should be the name of the pnode
|
||||
self.assertEqual(res.data['processing_node_name'], str(pnode))
|
||||
|
||||
# Can download assets
|
||||
for asset in list(task.ASSETS_MAP.keys()):
|
||||
|
|
|
@ -18,7 +18,7 @@ logger = logging.getLogger('app.logger')
|
|||
def start_processing_node(*args):
|
||||
current_dir = os.path.dirname(os.path.realpath(__file__))
|
||||
node_odm = subprocess.Popen(['node', 'index.js', '--port', '11223', '--test'] + list(args), shell=False,
|
||||
cwd=os.path.join(current_dir, "..", "..", "nodeodm", "external", "node-OpenDroneMap"))
|
||||
cwd=os.path.join(current_dir, "..", "..", "nodeodm", "external", "NodeODM"))
|
||||
time.sleep(2) # Wait for the server to launch
|
||||
return node_odm
|
||||
|
||||
|
|
|
@ -4,3 +4,6 @@ services:
|
|||
entrypoint: /bin/bash -c "chmod +x /webodm/*.sh && /bin/bash -c \"/webodm/wait-for-postgres.sh db /webodm/wait-for-it.sh -t 0 broker:6379 -- /webodm/start.sh --create-default-pnode --setup-devenv\""
|
||||
volumes:
|
||||
- .:/webodm
|
||||
worker:
|
||||
volumes:
|
||||
- .:/webodm
|
|
@ -9,7 +9,7 @@ services:
|
|||
depends_on:
|
||||
- node-odm-1
|
||||
node-odm-1:
|
||||
image: opendronemap/node-opendronemap
|
||||
image: opendronemap/nodeodm
|
||||
container_name: node-odm-1
|
||||
ports:
|
||||
- "3000"
|
||||
|
|
|
@ -1,7 +1,9 @@
|
|||
"""
|
||||
An interface to node-OpenDroneMap's API
|
||||
https://github.com/pierotofy/node-OpenDroneMap/blob/master/docs/index.adoc
|
||||
An interface to NodeODM's API
|
||||
https://github.com/pierotofy/NodeODM/blob/master/docs/index.adoc
|
||||
"""
|
||||
from requests.packages.urllib3.fields import RequestField
|
||||
from requests_toolbelt.multipart import encoder
|
||||
import requests
|
||||
import mimetypes
|
||||
import json
|
||||
|
@ -9,6 +11,47 @@ import os
|
|||
from urllib.parse import urlunparse, urlencode
|
||||
from app.testwatch import TestWatch
|
||||
|
||||
# Extends class to support multipart form data
|
||||
# fields with the same name
|
||||
# https://github.com/requests/toolbelt/issues/225
|
||||
class MultipartEncoder(encoder.MultipartEncoder):
|
||||
"""Multiple files with the same name support, i.e. files[]"""
|
||||
|
||||
def _iter_fields(self):
|
||||
_fields = self.fields
|
||||
if hasattr(self.fields, 'items'):
|
||||
_fields = list(self.fields.items())
|
||||
for k, v in _fields:
|
||||
for field in self._iter_field(k, v):
|
||||
yield field
|
||||
|
||||
@classmethod
|
||||
def _iter_field(cls, field_name, field):
|
||||
file_name = None
|
||||
file_type = None
|
||||
file_headers = None
|
||||
if field and isinstance(field, (list, tuple)):
|
||||
if all([isinstance(f, (list, tuple)) for f in field]):
|
||||
for f in field:
|
||||
yield next(cls._iter_field(field_name, f))
|
||||
else:
|
||||
raise StopIteration()
|
||||
if len(field) == 2:
|
||||
file_name, file_pointer = field
|
||||
elif len(field) == 3:
|
||||
file_name, file_pointer, file_type = field
|
||||
else:
|
||||
file_name, file_pointer, file_type, file_headers = field
|
||||
else:
|
||||
file_pointer = field
|
||||
|
||||
field = RequestField(name=field_name,
|
||||
data=file_pointer,
|
||||
filename=file_name,
|
||||
headers=file_headers)
|
||||
field.make_multipart(content_type=file_type)
|
||||
yield field
|
||||
|
||||
class ApiClient:
|
||||
def __init__(self, host, port, token = "", timeout=30):
|
||||
self.host = host
|
||||
|
@ -17,13 +60,13 @@ class ApiClient:
|
|||
self.timeout = timeout
|
||||
|
||||
def url(self, url, query = {}):
|
||||
netloc = self.host if self.port == 80 else "{}:{}".format(self.host, self.port)
|
||||
netloc = self.host if (self.port == 80 or self.port == 443) else "{}:{}".format(self.host, self.port)
|
||||
proto = 'https' if self.port == 443 else 'http'
|
||||
|
||||
if len(self.token) > 0:
|
||||
query['token'] = self.token
|
||||
|
||||
# TODO: https support
|
||||
return urlunparse(('http', netloc, url, '', urlencode(query), ''))
|
||||
return urlunparse((proto, netloc, url, '', urlencode(query), ''))
|
||||
|
||||
def info(self):
|
||||
return requests.get(self.url('/info'), timeout=self.timeout).json()
|
||||
|
@ -56,12 +99,13 @@ class ApiClient:
|
|||
else:
|
||||
return res
|
||||
|
||||
def new_task(self, images, name=None, options=[]):
|
||||
def new_task(self, images, name=None, options=[], progress_callback=None):
|
||||
"""
|
||||
Starts processing of a new task
|
||||
:param images: list of path images
|
||||
:param name: name of the task
|
||||
:param options: options to be used for processing ([{'name': optionName, 'value': optionValue}, ...])
|
||||
:param progress_callback: optional callback invoked during the upload images process to be used to report status.
|
||||
:return: UUID or error
|
||||
"""
|
||||
|
||||
|
@ -72,9 +116,24 @@ class ApiClient:
|
|||
with open(path, 'rb') as f:
|
||||
return f.read()
|
||||
|
||||
files = [('images',
|
||||
(os.path.basename(image), read_file(image), (mimetypes.guess_type(image)[0] or "image/jpg"))
|
||||
) for image in images]
|
||||
fields = {
|
||||
'name': name,
|
||||
'options': json.dumps(options),
|
||||
'images': [(os.path.basename(image), read_file(image), (mimetypes.guess_type(image)[0] or "image/jpg")) for image in images]
|
||||
}
|
||||
|
||||
def create_callback(mpe):
|
||||
total_bytes = mpe.len
|
||||
|
||||
def callback(monitor):
|
||||
if progress_callback is not None and total_bytes > 0:
|
||||
progress_callback(monitor.bytes_read / total_bytes)
|
||||
|
||||
return callback
|
||||
|
||||
e = MultipartEncoder(fields=fields)
|
||||
m = encoder.MultipartEncoderMonitor(e, create_callback(e))
|
||||
|
||||
return requests.post(self.url("/task/new"),
|
||||
files=files,
|
||||
data={'name': name, 'options': json.dumps(options)}).json()
|
||||
data=m,
|
||||
headers={'Content-Type': m.content_type}).json()
|
|
@ -0,0 +1 @@
|
|||
Subproject commit d0e1e1424e90c21fee401ce6e5a7dd1de7b32112
|
|
@ -1 +0,0 @@
|
|||
Subproject commit 1c9e149a7c32da438f7844b8cfb81bec74a050d4
|
|
@ -0,0 +1,18 @@
|
|||
# Generated by Django 2.0.3 on 2018-12-04 17:04
|
||||
|
||||
from django.db import migrations, models
|
||||
|
||||
|
||||
class Migration(migrations.Migration):
|
||||
|
||||
dependencies = [
|
||||
('nodeodm', '0003_auto_20180625_1230'),
|
||||
]
|
||||
|
||||
operations = [
|
||||
migrations.AddField(
|
||||
model_name='processingnode',
|
||||
name='max_images',
|
||||
field=models.PositiveIntegerField(blank=True, help_text='Maximum number of images accepted by this node.', null=True),
|
||||
),
|
||||
]
|
|
@ -42,6 +42,7 @@ class ProcessingNode(models.Model):
|
|||
queue_count = models.PositiveIntegerField(default=0, help_text="Number of tasks currently being processed by this node (as reported by the node itself)")
|
||||
available_options = fields.JSONField(default=dict(), help_text="Description of the options that can be used for processing")
|
||||
token = models.CharField(max_length=1024, blank=True, default="", help_text="Token to use for authentication. If the node doesn't have authentication, you can leave this field blank.")
|
||||
max_images = models.PositiveIntegerField(help_text="Maximum number of images accepted by this node.", blank=True, null=True)
|
||||
|
||||
def __str__(self):
|
||||
return '{}:{}'.format(self.hostname, self.port)
|
||||
|
@ -70,9 +71,15 @@ class ProcessingNode(models.Model):
|
|||
api_client = self.api_client(timeout=5)
|
||||
try:
|
||||
info = api_client.info()
|
||||
if 'error' in info:
|
||||
return False
|
||||
|
||||
self.api_version = info['version']
|
||||
self.queue_count = info['taskQueueCount']
|
||||
|
||||
if 'maxImages' in info:
|
||||
self.max_images = info['maxImages']
|
||||
|
||||
options = api_client.options()
|
||||
self.available_options = options
|
||||
self.last_refreshed = timezone.now()
|
||||
|
@ -92,7 +99,7 @@ class ProcessingNode(models.Model):
|
|||
return json.dumps(self.available_options, **kwargs)
|
||||
|
||||
@api
|
||||
def process_new_task(self, images, name=None, options=[]):
|
||||
def process_new_task(self, images, name=None, options=[], progress_callback=None):
|
||||
"""
|
||||
Sends a set of images (and optional GCP file) via the API
|
||||
to start processing.
|
||||
|
@ -100,6 +107,7 @@ class ProcessingNode(models.Model):
|
|||
:param images: list of path images
|
||||
:param name: name of the task
|
||||
:param options: options to be used for processing ([{'name': optionName, 'value': optionValue}, ...])
|
||||
:param progress_callback: optional callback invoked during the upload images process to be used to report status.
|
||||
|
||||
:returns UUID of the newly created task
|
||||
"""
|
||||
|
@ -107,7 +115,7 @@ class ProcessingNode(models.Model):
|
|||
|
||||
api_client = self.api_client()
|
||||
try:
|
||||
result = api_client.new_task(images, name, options)
|
||||
result = api_client.new_task(images, name, options, progress_callback)
|
||||
except requests.exceptions.ConnectionError as e:
|
||||
raise ProcessingError(e)
|
||||
|
||||
|
@ -145,7 +153,7 @@ class ProcessingNode(models.Model):
|
|||
if isinstance(result, dict) and 'error' in result:
|
||||
raise ProcessingError(result['error'])
|
||||
elif isinstance(result, list):
|
||||
return "\n".join(result)
|
||||
return result
|
||||
else:
|
||||
raise ProcessingError("Unknown response for console output: {}".format(result))
|
||||
|
||||
|
@ -189,7 +197,7 @@ class ProcessingNode(models.Model):
|
|||
def handle_generic_post_response(result):
|
||||
"""
|
||||
Handles a POST response that has either a "success" flag, or an error message.
|
||||
This is a common response in node-OpenDroneMap POST calls.
|
||||
This is a common response in NodeODM POST calls.
|
||||
:param result: result of API call
|
||||
:return: True on success, raises ProcessingException otherwise
|
||||
"""
|
||||
|
|
|
@ -21,7 +21,7 @@ class TestClientApi(TestCase):
|
|||
@classmethod
|
||||
def setUpClass(cls):
|
||||
super(TestClientApi, cls).setUpClass()
|
||||
cls.node_odm = subprocess.Popen(['node', 'index.js', '--port', '11223', '--test'], shell=False, cwd=path.join(current_dir, "external", "node-OpenDroneMap"))
|
||||
cls.node_odm = subprocess.Popen(['node', 'index.js', '--port', '11223', '--test'], shell=False, cwd=path.join(current_dir, "external", "NodeODM"))
|
||||
time.sleep(2) # Wait for the server to launch
|
||||
|
||||
|
||||
|
@ -45,6 +45,7 @@ class TestClientApi(TestCase):
|
|||
info = self.api_client.info()
|
||||
self.assertTrue(isinstance(info['version'], six.string_types), "Found version string")
|
||||
self.assertTrue(isinstance(info['taskQueueCount'], int), "Found task queue count")
|
||||
self.assertTrue(info['maxImages'] is None, "Found task max images")
|
||||
|
||||
def test_options(self):
|
||||
options = self.api_client.options()
|
||||
|
@ -58,9 +59,10 @@ class TestClientApi(TestCase):
|
|||
self.assertTrue(online_node.api_version == "", "API version is not set")
|
||||
|
||||
self.assertTrue(online_node.update_node_info(), "Could update info")
|
||||
self.assertTrue(online_node.last_refreshed != None, "Last refreshed is set")
|
||||
self.assertTrue(online_node.last_refreshed is not None, "Last refreshed is set")
|
||||
self.assertTrue(len(online_node.available_options) > 0, "Available options are set")
|
||||
self.assertTrue(online_node.api_version != "", "API version is set")
|
||||
self.assertTrue(online_node.max_images is None, "No max images limit is set")
|
||||
|
||||
self.assertTrue(isinstance(online_node.get_available_options_json(), six.string_types), "Available options json works")
|
||||
self.assertTrue(isinstance(online_node.get_available_options_json(pretty=True), six.string_types), "Available options json works with pretty")
|
||||
|
@ -121,7 +123,7 @@ class TestClientApi(TestCase):
|
|||
|
||||
# task_output
|
||||
self.assertTrue(isinstance(api.task_output(uuid, 0), list))
|
||||
self.assertTrue(isinstance(online_node.get_task_console_output(uuid, 0), str))
|
||||
self.assertTrue(isinstance(online_node.get_task_console_output(uuid, 0), list))
|
||||
|
||||
self.assertRaises(ProcessingError, online_node.get_task_console_output, "wrong-uuid", 0)
|
||||
|
||||
|
@ -157,6 +159,10 @@ class TestClientApi(TestCase):
|
|||
# Task has been deleted
|
||||
self.assertRaises(ProcessingError, online_node.get_task_info, uuid)
|
||||
|
||||
# Test URL building for HTTPS
|
||||
sslApi = ApiClient("localhost", 443, 'abc')
|
||||
self.assertEqual(sslApi.url('/info'), 'https://localhost/info?token=abc')
|
||||
|
||||
def test_find_best_available_node_and_is_online(self):
|
||||
# Fixtures are all offline
|
||||
self.assertTrue(ProcessingNode.find_best_available_node() is None)
|
||||
|
@ -191,7 +197,7 @@ class TestClientApi(TestCase):
|
|||
def test_token_auth(self):
|
||||
node_odm = subprocess.Popen(
|
||||
['node', 'index.js', '--port', '11224', '--token', 'test_token', '--test'], shell=False,
|
||||
cwd=path.join(current_dir, "external", "node-OpenDroneMap"))
|
||||
cwd=path.join(current_dir, "external", "NodeODM"))
|
||||
time.sleep(2)
|
||||
|
||||
def wait_for_status(api, uuid, status, num_retries=10, error_description="Failed to wait for status"):
|
||||
|
@ -215,10 +221,10 @@ class TestClientApi(TestCase):
|
|||
|
||||
self.assertTrue(online_node.update_node_info(), "Could update info")
|
||||
|
||||
# Can always call info(), options() (even without valid tokens)
|
||||
# Cannot call info(), options() without tokens
|
||||
api.token = "invalid"
|
||||
self.assertTrue(type(api.info()['version']) == str)
|
||||
self.assertTrue(len(api.options()) > 0)
|
||||
self.assertTrue(type(api.info()['error']) == str)
|
||||
self.assertTrue(type(api.options()['error']) == str)
|
||||
|
||||
# Cannot call new_task() without token
|
||||
import glob
|
||||
|
|
|
@ -1,6 +1,6 @@
|
|||
{
|
||||
"name": "WebODM",
|
||||
"version": "0.6.2",
|
||||
"version": "0.7.0",
|
||||
"description": "Open Source Drone Image Processing",
|
||||
"main": "index.js",
|
||||
"scripts": {
|
||||
|
|
|
@ -41,6 +41,7 @@ pyparsing==2.1.10
|
|||
pytz==2018.3
|
||||
rcssmin==1.0.6
|
||||
redis==2.10.6
|
||||
requests-toolbelt==0.8.0
|
||||
requests==2.20.0
|
||||
rfc3987==1.3.7
|
||||
rjsmin==1.0.12
|
||||
|
|
|
@ -51,7 +51,7 @@ Directories of interest are listed as follow:
|
|||
Directory | Description
|
||||
--------- | -----------
|
||||
`/app` | Main application, includes the UI components, API, tests and backend logic.
|
||||
`/nodeodm`| Application that bridges the communication between WebODM and [node-OpenDroneMap](https://github.com/OpenDroneMap/node-OpenDroneMap). Includes its own unit tests and models.
|
||||
`/nodeodm`| Application that bridges the communication between WebODM and [NodeODM](https://github.com/OpenDroneMap/NodeODM). Includes its own unit tests and models.
|
||||
`/webodm` | Django's main project directory. Setting files are here.
|
||||
|
||||
### Frontend
|
||||
|
|
|
@ -11,6 +11,7 @@
|
|||
"api_version": "1.0.1",
|
||||
"last_refreshed": "2017-03-01T21:14:49.918276Z",
|
||||
"queue_count": 0,
|
||||
"max_images": null,
|
||||
"available_options": [
|
||||
{
|
||||
"help": "Oct-tree depth at which the Laplacian equation is solved in the surface reconstruction step. Increasing this value increases computation times slightly but helps reduce memory usage. Default: 9",
|
||||
|
@ -23,7 +24,7 @@
|
|||
```
|
||||
|
||||
Processing nodes are associated with zero or more tasks and
|
||||
take care of processing input images. Processing nodes are computers or virtual machines running [node-OpenDroneMap](https://github.com/OpenDroneMap/node-OpenDroneMap/) or any other API compatible with it.
|
||||
take care of processing input images. Processing nodes are computers or virtual machines running [NodeODM](https://github.com/OpenDroneMap/NodeODM) or any other API compatible with it.
|
||||
|
||||
Field | Type | Description
|
||||
----- | ---- | -----------
|
||||
|
@ -31,9 +32,10 @@ id | int | Unique Identifier
|
|||
online | bool | Whether the processing node could be reached in the last 5 minutes
|
||||
hostname | string | Hostname/IP address
|
||||
port | int | Port
|
||||
api_version | string | Version of node-OpenDroneMap currently running
|
||||
api_version | string | Version of NodeODM currently running
|
||||
last_refreshed | string | Date and time this node was last seen online. This value is typically refreshed every 15-30 seconds and is used to decide whether a node is offline or not
|
||||
queue_count | int | Number of [Task](#task) items currently being processed/queued on this node.
|
||||
max_images | int | Optional maximum number of images this processing node can accept. null indicates no limit.
|
||||
available_options | JSON[] | JSON-encoded list of options that this node is capable of handling. See [Available Options](#available-options) for more information
|
||||
|
||||
|
||||
|
@ -45,7 +47,7 @@ help | Description of the option
|
|||
name | Name that identifies the option. This is the value you pass in the `name` key/value pair when creating a set of options for a new [Task](#task)
|
||||
type | Possible values are `int`, `float`, `string`, `bool`
|
||||
value | Default value if the option is not specified
|
||||
domain | Restriction of the range of values that this option allows. Examples are `float`, `negative integer`, `percent`, `float: 0 <= x <= 10`, etc. for all possible values, check [node-OpenDroneMap's odmOptions.js code](https://github.com/OpenDroneMap/node-OpenDroneMap/blob/master/libs/odmOptions.js#L135)
|
||||
domain | Restriction of the range of values that this option allows. Examples are `float`, `negative integer`, `percent`, `float: 0 <= x <= 10`, etc. for all possible values, check [NodeODM's odmOptions.js code](https://github.com/OpenDroneMap/NodeODM/blob/master/libs/odmOptions.js#L135)
|
||||
|
||||
|
||||
### Add a processing node
|
||||
|
@ -83,6 +85,7 @@ hostname | | "" | Filter by hostname
|
|||
port | | "" | Filter by port
|
||||
api_version | | "" | Filter by API version
|
||||
queue_count | | "" | Filter by queue count
|
||||
max_images | | "" | Filter by max images
|
||||
ordering | | "" | Ordering field to sort results by
|
||||
has_available_options | | "" | Return only processing nodes that have a valid set of processing options (check that the `available_options` field is populated). Either `true` or `false`.
|
||||
|
||||
|
@ -128,5 +131,5 @@ Display the common options available among all online processing nodes. This is
|
|||
|
||||
Use this list of options to check whether a particular option is supported by all online processing nodes. If you use the automatic processing node assignment feature for processing tasks, this is the list you want to display to the user for choosing the options to use during processing.
|
||||
|
||||
<aside class="notice">While WebODM is capable of handling processing nodes running different versions of node-OpenDroneMap, we don't recommend doing so. When all processing nodes use the same node-OpenDroneMap version, the output of this API call will be identical to the <b>available_options</b> field of any node.</aside>
|
||||
<aside class="notice">While WebODM is capable of handling processing nodes running different versions of NodeODM, we don't recommend doing so. When all processing nodes use the same NodeODM version, the output of this API call will be identical to the <b>available_options</b> field of any node.</aside>
|
||||
|
||||
|
|
|
@ -7,6 +7,7 @@
|
|||
"id": 134,
|
||||
"project": 27,
|
||||
"processing_node": 10,
|
||||
"processing_node_name": "localhost:3000",
|
||||
"images_count": 48,
|
||||
"can_rerun_from": [],
|
||||
"available_assets": [
|
||||
|
@ -30,9 +31,11 @@
|
|||
"value": true
|
||||
}
|
||||
],
|
||||
"ground_control_points": null,
|
||||
"created_at": "2017-02-18T18:01:55.402551Z",
|
||||
"pending_action": null
|
||||
"pending_action": null,
|
||||
"upload_progress": 1.0,
|
||||
"resize_progress": 0.0,
|
||||
"running_progress": 1.0
|
||||
}
|
||||
```
|
||||
|
||||
|
@ -43,6 +46,7 @@ Field | Type | Description
|
|||
id | int | Unique identifier
|
||||
project | int | [Project](#project) ID the task belongs to
|
||||
processing_node | int | The ID of the [Processing Node](#processing-node) this task has been assigned to, or `null` if no [Processing Node](#processing-node) has been assigned.
|
||||
processing_node_name | string | The name of the processing node below, or `null` if no [Processing Node](#processing-node) has been assigned.
|
||||
images_count | int | Number of images
|
||||
can_rerun_from | string[] | List of possible "rerun-from" options that this task could restart from, given its currently assigned processing node. If this is an empty list, the task can only be restarted from the start of the pipeline.
|
||||
available_assets | string[] | List of [assets](#download-assets) available for download
|
||||
|
@ -53,9 +57,12 @@ auto_processing_node | boolean | Whether WebODM should automatically assign the
|
|||
status | int | One of [Status Codes](#status-codes), or `null` if no status is available.
|
||||
last_error | string | The last error message reported by a [Processing Node](#processing-node) in case of processing failure.
|
||||
options | JSON[] | JSON-encoded list of name/value pairs, where each pair represents a command line option to be passed to a [Processing Node](#processing-node).
|
||||
ground_control_points | string | Currently unused. See [#37](https://github.com/OpenDroneMap/WebODM/issues/37)
|
||||
created_at | string | Creation date and time
|
||||
created_at | string | Creation date and time.
|
||||
pending_action | int | One of [Pending Actions](#pending-actions), or `null` if no pending action is set.
|
||||
upload_progress | float | Value between 0 and 1 indicating the upload progress of this task's files to the processing node.
|
||||
resize_progress | float | Value between 0 and 1 indicating the resize progress of this task's images.
|
||||
running_progress | float | Value between 0 and 1 indicating the running progress (estimated) of this task.
|
||||
|
||||
|
||||
<aside class="notice">Tasks inherit the permission settings from the <a href="#project">Project</a> they belong to.</aside>
|
||||
|
||||
|
@ -91,6 +98,7 @@ Parameters are the same as above.
|
|||
"id": 6,
|
||||
"project": 2,
|
||||
"processing_node": 2,
|
||||
"processing_node_name": "localhost:3000",
|
||||
"images_count": 89,
|
||||
"uuid": "2e8b687d-c269-4e2f-91b3-5a2cd51b5321",
|
||||
"name": "Test name",
|
||||
|
@ -99,9 +107,11 @@ Parameters are the same as above.
|
|||
"status": 40,
|
||||
"last_error": null,
|
||||
"options": [],
|
||||
"ground_control_points": null,
|
||||
"created_at": "2016-12-08T13:32:28.139474Z",
|
||||
"pending_action": null
|
||||
"pending_action": null,
|
||||
"upload_progress": 1.0,
|
||||
"resize_progress": 0.0,
|
||||
"running_progress": 1.0
|
||||
}
|
||||
]
|
||||
```
|
||||
|
|
|
@ -6,8 +6,8 @@ language_tabs:
|
|||
|
||||
toc_footers:
|
||||
- <a href='https://github.com/OpenDroneMap/WebODM'>WebODM on GitHub</a>
|
||||
- <a href='https://github.com/OpenDroneMap/OpenDroneMap'>OpenDroneMap on GitHub</a>
|
||||
- <a href='https://github.com/OpenDroneMap/node-OpenDroneMap'>node-OpenDroneMap on GitHub</a>
|
||||
- <a href='https://github.com/OpenDroneMap/ODM'>ODM on GitHub</a>
|
||||
- <a href='https://github.com/OpenDroneMap/NodeODM'>NodeODM on GitHub</a>
|
||||
|
||||
search: true
|
||||
|
||||
|
|
5
start.sh
5
start.sh
|
@ -43,10 +43,13 @@ if [ "$1" = "--setup-devenv" ] || [ "$2" = "--setup-devenv" ]; then
|
|||
|
||||
echo Setup npm dependencies...
|
||||
npm install
|
||||
cd nodeodm/external/node-OpenDroneMap
|
||||
cd nodeodm/external/NodeODM
|
||||
npm install
|
||||
cd /webodm
|
||||
|
||||
echo Setup pip requirements...
|
||||
pip install -r requirements.txt
|
||||
|
||||
echo Setup webpack watch...
|
||||
webpack --watch &
|
||||
fi
|
||||
|
|
|
@ -248,7 +248,7 @@ rebuild(){
|
|||
run "docker-compose down --remove-orphans"
|
||||
plugin_cleanup
|
||||
run "rm -fr node_modules/ || sudo rm -fr node_modules/"
|
||||
run "rm -fr nodeodm/external/node-OpenDroneMap || sudo rm -fr nodeodm/external/node-OpenDroneMap"
|
||||
run "rm -fr nodeodm/external/NodeODM || sudo rm -fr nodeodm/external/NodeODM"
|
||||
run "docker-compose -f docker-compose.yml -f docker-compose.build.yml build --no-cache"
|
||||
#run "docker images --no-trunc -aqf \"dangling=true\" | xargs docker rmi"
|
||||
echo -e "\033[1mDone!\033[0m You can now start WebODM by running $0 start"
|
||||
|
@ -371,7 +371,7 @@ elif [[ $1 = "rebuild" ]]; then
|
|||
elif [[ $1 = "update" ]]; then
|
||||
echo "Updating WebODM..."
|
||||
run "git pull origin master"
|
||||
run "docker pull opendronemap/node-opendronemap"
|
||||
run "docker pull opendronemap/nodeodm"
|
||||
run "docker pull opendronemap/webodm_db"
|
||||
run "docker pull opendronemap/webodm_webapp"
|
||||
run "docker-compose down --remove-orphans"
|
||||
|
|
Ładowanie…
Reference in New Issue