kopia lustrzana https://github.com/ukhas/habitat-cpp-connector
Move cpp-connector to its own Git repository.
See old habitat-extensions for previous history.pull/1/merge
Adam Greig
commit
b18975e4c4
|
|
@ -0,0 +1,4 @@
|
|||
cpp_connector
|
||||
cpp_connector_threaded
|
||||
extractor
|
||||
*.o
|
||||
|
|
@ -0,0 +1,675 @@
|
|||
GNU GENERAL PUBLIC LICENSE
|
||||
Version 3, 29 June 2007
|
||||
|
||||
Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
|
||||
Everyone is permitted to copy and distribute verbatim copies
|
||||
of this license document, but changing it is not allowed.
|
||||
|
||||
Preamble
|
||||
|
||||
The GNU General Public License is a free, copyleft license for
|
||||
software and other kinds of works.
|
||||
|
||||
The licenses for most software and other practical works are designed
|
||||
to take away your freedom to share and change the works. By contrast,
|
||||
the GNU General Public License is intended to guarantee your freedom to
|
||||
share and change all versions of a program--to make sure it remains free
|
||||
software for all its users. We, the Free Software Foundation, use the
|
||||
GNU General Public License for most of our software; it applies also to
|
||||
any other work released this way by its authors. You can apply it to
|
||||
your programs, too.
|
||||
|
||||
When we speak of free software, we are referring to freedom, not
|
||||
price. Our General Public Licenses are designed to make sure that you
|
||||
have the freedom to distribute copies of free software (and charge for
|
||||
them if you wish), that you receive source code or can get it if you
|
||||
want it, that you can change the software or use pieces of it in new
|
||||
free programs, and that you know you can do these things.
|
||||
|
||||
To protect your rights, we need to prevent others from denying you
|
||||
these rights or asking you to surrender the rights. Therefore, you have
|
||||
certain responsibilities if you distribute copies of the software, or if
|
||||
you modify it: responsibilities to respect the freedom of others.
|
||||
|
||||
For example, if you distribute copies of such a program, whether
|
||||
gratis or for a fee, you must pass on to the recipients the same
|
||||
freedoms that you received. You must make sure that they, too, receive
|
||||
or can get the source code. And you must show them these terms so they
|
||||
know their rights.
|
||||
|
||||
Developers that use the GNU GPL protect your rights with two steps:
|
||||
(1) assert copyright on the software, and (2) offer you this License
|
||||
giving you legal permission to copy, distribute and/or modify it.
|
||||
|
||||
For the developers' and authors' protection, the GPL clearly explains
|
||||
that there is no warranty for this free software. For both users' and
|
||||
authors' sake, the GPL requires that modified versions be marked as
|
||||
changed, so that their problems will not be attributed erroneously to
|
||||
authors of previous versions.
|
||||
|
||||
Some devices are designed to deny users access to install or run
|
||||
modified versions of the software inside them, although the manufacturer
|
||||
can do so. This is fundamentally incompatible with the aim of
|
||||
protecting users' freedom to change the software. The systematic
|
||||
pattern of such abuse occurs in the area of products for individuals to
|
||||
use, which is precisely where it is most unacceptable. Therefore, we
|
||||
have designed this version of the GPL to prohibit the practice for those
|
||||
products. If such problems arise substantially in other domains, we
|
||||
stand ready to extend this provision to those domains in future versions
|
||||
of the GPL, as needed to protect the freedom of users.
|
||||
|
||||
Finally, every program is threatened constantly by software patents.
|
||||
States should not allow patents to restrict development and use of
|
||||
software on general-purpose computers, but in those that do, we wish to
|
||||
avoid the special danger that patents applied to a free program could
|
||||
make it effectively proprietary. To prevent this, the GPL assures that
|
||||
patents cannot be used to render the program non-free.
|
||||
|
||||
The precise terms and conditions for copying, distribution and
|
||||
modification follow.
|
||||
|
||||
TERMS AND CONDITIONS
|
||||
|
||||
0. Definitions.
|
||||
|
||||
"This License" refers to version 3 of the GNU General Public License.
|
||||
|
||||
"Copyright" also means copyright-like laws that apply to other kinds of
|
||||
works, such as semiconductor masks.
|
||||
|
||||
"The Program" refers to any copyrightable work licensed under this
|
||||
License. Each licensee is addressed as "you". "Licensees" and
|
||||
"recipients" may be individuals or organizations.
|
||||
|
||||
To "modify" a work means to copy from or adapt all or part of the work
|
||||
in a fashion requiring copyright permission, other than the making of an
|
||||
exact copy. The resulting work is called a "modified version" of the
|
||||
earlier work or a work "based on" the earlier work.
|
||||
|
||||
A "covered work" means either the unmodified Program or a work based
|
||||
on the Program.
|
||||
|
||||
To "propagate" a work means to do anything with it that, without
|
||||
permission, would make you directly or secondarily liable for
|
||||
infringement under applicable copyright law, except executing it on a
|
||||
computer or modifying a private copy. Propagation includes copying,
|
||||
distribution (with or without modification), making available to the
|
||||
public, and in some countries other activities as well.
|
||||
|
||||
To "convey" a work means any kind of propagation that enables other
|
||||
parties to make or receive copies. Mere interaction with a user through
|
||||
a computer network, with no transfer of a copy, is not conveying.
|
||||
|
||||
An interactive user interface displays "Appropriate Legal Notices"
|
||||
to the extent that it includes a convenient and prominently visible
|
||||
feature that (1) displays an appropriate copyright notice, and (2)
|
||||
tells the user that there is no warranty for the work (except to the
|
||||
extent that warranties are provided), that licensees may convey the
|
||||
work under this License, and how to view a copy of this License. If
|
||||
the interface presents a list of user commands or options, such as a
|
||||
menu, a prominent item in the list meets this criterion.
|
||||
|
||||
1. Source Code.
|
||||
|
||||
The "source code" for a work means the preferred form of the work
|
||||
for making modifications to it. "Object code" means any non-source
|
||||
form of a work.
|
||||
|
||||
A "Standard Interface" means an interface that either is an official
|
||||
standard defined by a recognized standards body, or, in the case of
|
||||
interfaces specified for a particular programming language, one that
|
||||
is widely used among developers working in that language.
|
||||
|
||||
The "System Libraries" of an executable work include anything, other
|
||||
than the work as a whole, that (a) is included in the normal form of
|
||||
packaging a Major Component, but which is not part of that Major
|
||||
Component, and (b) serves only to enable use of the work with that
|
||||
Major Component, or to implement a Standard Interface for which an
|
||||
implementation is available to the public in source code form. A
|
||||
"Major Component", in this context, means a major essential component
|
||||
(kernel, window system, and so on) of the specific operating system
|
||||
(if any) on which the executable work runs, or a compiler used to
|
||||
produce the work, or an object code interpreter used to run it.
|
||||
|
||||
The "Corresponding Source" for a work in object code form means all
|
||||
the source code needed to generate, install, and (for an executable
|
||||
work) run the object code and to modify the work, including scripts to
|
||||
control those activities. However, it does not include the work's
|
||||
System Libraries, or general-purpose tools or generally available free
|
||||
programs which are used unmodified in performing those activities but
|
||||
which are not part of the work. For example, Corresponding Source
|
||||
includes interface definition files associated with source files for
|
||||
the work, and the source code for shared libraries and dynamically
|
||||
linked subprograms that the work is specifically designed to require,
|
||||
such as by intimate data communication or control flow between those
|
||||
subprograms and other parts of the work.
|
||||
|
||||
The Corresponding Source need not include anything that users
|
||||
can regenerate automatically from other parts of the Corresponding
|
||||
Source.
|
||||
|
||||
The Corresponding Source for a work in source code form is that
|
||||
same work.
|
||||
|
||||
2. Basic Permissions.
|
||||
|
||||
All rights granted under this License are granted for the term of
|
||||
copyright on the Program, and are irrevocable provided the stated
|
||||
conditions are met. This License explicitly affirms your unlimited
|
||||
permission to run the unmodified Program. The output from running a
|
||||
covered work is covered by this License only if the output, given its
|
||||
content, constitutes a covered work. This License acknowledges your
|
||||
rights of fair use or other equivalent, as provided by copyright law.
|
||||
|
||||
You may make, run and propagate covered works that you do not
|
||||
convey, without conditions so long as your license otherwise remains
|
||||
in force. You may convey covered works to others for the sole purpose
|
||||
of having them make modifications exclusively for you, or provide you
|
||||
with facilities for running those works, provided that you comply with
|
||||
the terms of this License in conveying all material for which you do
|
||||
not control copyright. Those thus making or running the covered works
|
||||
for you must do so exclusively on your behalf, under your direction
|
||||
and control, on terms that prohibit them from making any copies of
|
||||
your copyrighted material outside their relationship with you.
|
||||
|
||||
Conveying under any other circumstances is permitted solely under
|
||||
the conditions stated below. Sublicensing is not allowed; section 10
|
||||
makes it unnecessary.
|
||||
|
||||
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
|
||||
|
||||
No covered work shall be deemed part of an effective technological
|
||||
measure under any applicable law fulfilling obligations under article
|
||||
11 of the WIPO copyright treaty adopted on 20 December 1996, or
|
||||
similar laws prohibiting or restricting circumvention of such
|
||||
measures.
|
||||
|
||||
When you convey a covered work, you waive any legal power to forbid
|
||||
circumvention of technological measures to the extent such circumvention
|
||||
is effected by exercising rights under this License with respect to
|
||||
the covered work, and you disclaim any intention to limit operation or
|
||||
modification of the work as a means of enforcing, against the work's
|
||||
users, your or third parties' legal rights to forbid circumvention of
|
||||
technological measures.
|
||||
|
||||
4. Conveying Verbatim Copies.
|
||||
|
||||
You may convey verbatim copies of the Program's source code as you
|
||||
receive it, in any medium, provided that you conspicuously and
|
||||
appropriately publish on each copy an appropriate copyright notice;
|
||||
keep intact all notices stating that this License and any
|
||||
non-permissive terms added in accord with section 7 apply to the code;
|
||||
keep intact all notices of the absence of any warranty; and give all
|
||||
recipients a copy of this License along with the Program.
|
||||
|
||||
You may charge any price or no price for each copy that you convey,
|
||||
and you may offer support or warranty protection for a fee.
|
||||
|
||||
5. Conveying Modified Source Versions.
|
||||
|
||||
You may convey a work based on the Program, or the modifications to
|
||||
produce it from the Program, in the form of source code under the
|
||||
terms of section 4, provided that you also meet all of these conditions:
|
||||
|
||||
a) The work must carry prominent notices stating that you modified
|
||||
it, and giving a relevant date.
|
||||
|
||||
b) The work must carry prominent notices stating that it is
|
||||
released under this License and any conditions added under section
|
||||
7. This requirement modifies the requirement in section 4 to
|
||||
"keep intact all notices".
|
||||
|
||||
c) You must license the entire work, as a whole, under this
|
||||
License to anyone who comes into possession of a copy. This
|
||||
License will therefore apply, along with any applicable section 7
|
||||
additional terms, to the whole of the work, and all its parts,
|
||||
regardless of how they are packaged. This License gives no
|
||||
permission to license the work in any other way, but it does not
|
||||
invalidate such permission if you have separately received it.
|
||||
|
||||
d) If the work has interactive user interfaces, each must display
|
||||
Appropriate Legal Notices; however, if the Program has interactive
|
||||
interfaces that do not display Appropriate Legal Notices, your
|
||||
work need not make them do so.
|
||||
|
||||
A compilation of a covered work with other separate and independent
|
||||
works, which are not by their nature extensions of the covered work,
|
||||
and which are not combined with it such as to form a larger program,
|
||||
in or on a volume of a storage or distribution medium, is called an
|
||||
"aggregate" if the compilation and its resulting copyright are not
|
||||
used to limit the access or legal rights of the compilation's users
|
||||
beyond what the individual works permit. Inclusion of a covered work
|
||||
in an aggregate does not cause this License to apply to the other
|
||||
parts of the aggregate.
|
||||
|
||||
6. Conveying Non-Source Forms.
|
||||
|
||||
You may convey a covered work in object code form under the terms
|
||||
of sections 4 and 5, provided that you also convey the
|
||||
machine-readable Corresponding Source under the terms of this License,
|
||||
in one of these ways:
|
||||
|
||||
a) Convey the object code in, or embodied in, a physical product
|
||||
(including a physical distribution medium), accompanied by the
|
||||
Corresponding Source fixed on a durable physical medium
|
||||
customarily used for software interchange.
|
||||
|
||||
b) Convey the object code in, or embodied in, a physical product
|
||||
(including a physical distribution medium), accompanied by a
|
||||
written offer, valid for at least three years and valid for as
|
||||
long as you offer spare parts or customer support for that product
|
||||
model, to give anyone who possesses the object code either (1) a
|
||||
copy of the Corresponding Source for all the software in the
|
||||
product that is covered by this License, on a durable physical
|
||||
medium customarily used for software interchange, for a price no
|
||||
more than your reasonable cost of physically performing this
|
||||
conveying of source, or (2) access to copy the
|
||||
Corresponding Source from a network server at no charge.
|
||||
|
||||
c) Convey individual copies of the object code with a copy of the
|
||||
written offer to provide the Corresponding Source. This
|
||||
alternative is allowed only occasionally and noncommercially, and
|
||||
only if you received the object code with such an offer, in accord
|
||||
with subsection 6b.
|
||||
|
||||
d) Convey the object code by offering access from a designated
|
||||
place (gratis or for a charge), and offer equivalent access to the
|
||||
Corresponding Source in the same way through the same place at no
|
||||
further charge. You need not require recipients to copy the
|
||||
Corresponding Source along with the object code. If the place to
|
||||
copy the object code is a network server, the Corresponding Source
|
||||
may be on a different server (operated by you or a third party)
|
||||
that supports equivalent copying facilities, provided you maintain
|
||||
clear directions next to the object code saying where to find the
|
||||
Corresponding Source. Regardless of what server hosts the
|
||||
Corresponding Source, you remain obligated to ensure that it is
|
||||
available for as long as needed to satisfy these requirements.
|
||||
|
||||
e) Convey the object code using peer-to-peer transmission, provided
|
||||
you inform other peers where the object code and Corresponding
|
||||
Source of the work are being offered to the general public at no
|
||||
charge under subsection 6d.
|
||||
|
||||
A separable portion of the object code, whose source code is excluded
|
||||
from the Corresponding Source as a System Library, need not be
|
||||
included in conveying the object code work.
|
||||
|
||||
A "User Product" is either (1) a "consumer product", which means any
|
||||
tangible personal property which is normally used for personal, family,
|
||||
or household purposes, or (2) anything designed or sold for incorporation
|
||||
into a dwelling. In determining whether a product is a consumer product,
|
||||
doubtful cases shall be resolved in favor of coverage. For a particular
|
||||
product received by a particular user, "normally used" refers to a
|
||||
typical or common use of that class of product, regardless of the status
|
||||
of the particular user or of the way in which the particular user
|
||||
actually uses, or expects or is expected to use, the product. A product
|
||||
is a consumer product regardless of whether the product has substantial
|
||||
commercial, industrial or non-consumer uses, unless such uses represent
|
||||
the only significant mode of use of the product.
|
||||
|
||||
"Installation Information" for a User Product means any methods,
|
||||
procedures, authorization keys, or other information required to install
|
||||
and execute modified versions of a covered work in that User Product from
|
||||
a modified version of its Corresponding Source. The information must
|
||||
suffice to ensure that the continued functioning of the modified object
|
||||
code is in no case prevented or interfered with solely because
|
||||
modification has been made.
|
||||
|
||||
If you convey an object code work under this section in, or with, or
|
||||
specifically for use in, a User Product, and the conveying occurs as
|
||||
part of a transaction in which the right of possession and use of the
|
||||
User Product is transferred to the recipient in perpetuity or for a
|
||||
fixed term (regardless of how the transaction is characterized), the
|
||||
Corresponding Source conveyed under this section must be accompanied
|
||||
by the Installation Information. But this requirement does not apply
|
||||
if neither you nor any third party retains the ability to install
|
||||
modified object code on the User Product (for example, the work has
|
||||
been installed in ROM).
|
||||
|
||||
The requirement to provide Installation Information does not include a
|
||||
requirement to continue to provide support service, warranty, or updates
|
||||
for a work that has been modified or installed by the recipient, or for
|
||||
the User Product in which it has been modified or installed. Access to a
|
||||
network may be denied when the modification itself materially and
|
||||
adversely affects the operation of the network or violates the rules and
|
||||
protocols for communication across the network.
|
||||
|
||||
Corresponding Source conveyed, and Installation Information provided,
|
||||
in accord with this section must be in a format that is publicly
|
||||
documented (and with an implementation available to the public in
|
||||
source code form), and must require no special password or key for
|
||||
unpacking, reading or copying.
|
||||
|
||||
7. Additional Terms.
|
||||
|
||||
"Additional permissions" are terms that supplement the terms of this
|
||||
License by making exceptions from one or more of its conditions.
|
||||
Additional permissions that are applicable to the entire Program shall
|
||||
be treated as though they were included in this License, to the extent
|
||||
that they are valid under applicable law. If additional permissions
|
||||
apply only to part of the Program, that part may be used separately
|
||||
under those permissions, but the entire Program remains governed by
|
||||
this License without regard to the additional permissions.
|
||||
|
||||
When you convey a copy of a covered work, you may at your option
|
||||
remove any additional permissions from that copy, or from any part of
|
||||
it. (Additional permissions may be written to require their own
|
||||
removal in certain cases when you modify the work.) You may place
|
||||
additional permissions on material, added by you to a covered work,
|
||||
for which you have or can give appropriate copyright permission.
|
||||
|
||||
Notwithstanding any other provision of this License, for material you
|
||||
add to a covered work, you may (if authorized by the copyright holders of
|
||||
that material) supplement the terms of this License with terms:
|
||||
|
||||
a) Disclaiming warranty or limiting liability differently from the
|
||||
terms of sections 15 and 16 of this License; or
|
||||
|
||||
b) Requiring preservation of specified reasonable legal notices or
|
||||
author attributions in that material or in the Appropriate Legal
|
||||
Notices displayed by works containing it; or
|
||||
|
||||
c) Prohibiting misrepresentation of the origin of that material, or
|
||||
requiring that modified versions of such material be marked in
|
||||
reasonable ways as different from the original version; or
|
||||
|
||||
d) Limiting the use for publicity purposes of names of licensors or
|
||||
authors of the material; or
|
||||
|
||||
e) Declining to grant rights under trademark law for use of some
|
||||
trade names, trademarks, or service marks; or
|
||||
|
||||
f) Requiring indemnification of licensors and authors of that
|
||||
material by anyone who conveys the material (or modified versions of
|
||||
it) with contractual assumptions of liability to the recipient, for
|
||||
any liability that these contractual assumptions directly impose on
|
||||
those licensors and authors.
|
||||
|
||||
All other non-permissive additional terms are considered "further
|
||||
restrictions" within the meaning of section 10. If the Program as you
|
||||
received it, or any part of it, contains a notice stating that it is
|
||||
governed by this License along with a term that is a further
|
||||
restriction, you may remove that term. If a license document contains
|
||||
a further restriction but permits relicensing or conveying under this
|
||||
License, you may add to a covered work material governed by the terms
|
||||
of that license document, provided that the further restriction does
|
||||
not survive such relicensing or conveying.
|
||||
|
||||
If you add terms to a covered work in accord with this section, you
|
||||
must place, in the relevant source files, a statement of the
|
||||
additional terms that apply to those files, or a notice indicating
|
||||
where to find the applicable terms.
|
||||
|
||||
Additional terms, permissive or non-permissive, may be stated in the
|
||||
form of a separately written license, or stated as exceptions;
|
||||
the above requirements apply either way.
|
||||
|
||||
8. Termination.
|
||||
|
||||
You may not propagate or modify a covered work except as expressly
|
||||
provided under this License. Any attempt otherwise to propagate or
|
||||
modify it is void, and will automatically terminate your rights under
|
||||
this License (including any patent licenses granted under the third
|
||||
paragraph of section 11).
|
||||
|
||||
However, if you cease all violation of this License, then your
|
||||
license from a particular copyright holder is reinstated (a)
|
||||
provisionally, unless and until the copyright holder explicitly and
|
||||
finally terminates your license, and (b) permanently, if the copyright
|
||||
holder fails to notify you of the violation by some reasonable means
|
||||
prior to 60 days after the cessation.
|
||||
|
||||
Moreover, your license from a particular copyright holder is
|
||||
reinstated permanently if the copyright holder notifies you of the
|
||||
violation by some reasonable means, this is the first time you have
|
||||
received notice of violation of this License (for any work) from that
|
||||
copyright holder, and you cure the violation prior to 30 days after
|
||||
your receipt of the notice.
|
||||
|
||||
Termination of your rights under this section does not terminate the
|
||||
licenses of parties who have received copies or rights from you under
|
||||
this License. If your rights have been terminated and not permanently
|
||||
reinstated, you do not qualify to receive new licenses for the same
|
||||
material under section 10.
|
||||
|
||||
9. Acceptance Not Required for Having Copies.
|
||||
|
||||
You are not required to accept this License in order to receive or
|
||||
run a copy of the Program. Ancillary propagation of a covered work
|
||||
occurring solely as a consequence of using peer-to-peer transmission
|
||||
to receive a copy likewise does not require acceptance. However,
|
||||
nothing other than this License grants you permission to propagate or
|
||||
modify any covered work. These actions infringe copyright if you do
|
||||
not accept this License. Therefore, by modifying or propagating a
|
||||
covered work, you indicate your acceptance of this License to do so.
|
||||
|
||||
10. Automatic Licensing of Downstream Recipients.
|
||||
|
||||
Each time you convey a covered work, the recipient automatically
|
||||
receives a license from the original licensors, to run, modify and
|
||||
propagate that work, subject to this License. You are not responsible
|
||||
for enforcing compliance by third parties with this License.
|
||||
|
||||
An "entity transaction" is a transaction transferring control of an
|
||||
organization, or substantially all assets of one, or subdividing an
|
||||
organization, or merging organizations. If propagation of a covered
|
||||
work results from an entity transaction, each party to that
|
||||
transaction who receives a copy of the work also receives whatever
|
||||
licenses to the work the party's predecessor in interest had or could
|
||||
give under the previous paragraph, plus a right to possession of the
|
||||
Corresponding Source of the work from the predecessor in interest, if
|
||||
the predecessor has it or can get it with reasonable efforts.
|
||||
|
||||
You may not impose any further restrictions on the exercise of the
|
||||
rights granted or affirmed under this License. For example, you may
|
||||
not impose a license fee, royalty, or other charge for exercise of
|
||||
rights granted under this License, and you may not initiate litigation
|
||||
(including a cross-claim or counterclaim in a lawsuit) alleging that
|
||||
any patent claim is infringed by making, using, selling, offering for
|
||||
sale, or importing the Program or any portion of it.
|
||||
|
||||
11. Patents.
|
||||
|
||||
A "contributor" is a copyright holder who authorizes use under this
|
||||
License of the Program or a work on which the Program is based. The
|
||||
work thus licensed is called the contributor's "contributor version".
|
||||
|
||||
A contributor's "essential patent claims" are all patent claims
|
||||
owned or controlled by the contributor, whether already acquired or
|
||||
hereafter acquired, that would be infringed by some manner, permitted
|
||||
by this License, of making, using, or selling its contributor version,
|
||||
but do not include claims that would be infringed only as a
|
||||
consequence of further modification of the contributor version. For
|
||||
purposes of this definition, "control" includes the right to grant
|
||||
patent sublicenses in a manner consistent with the requirements of
|
||||
this License.
|
||||
|
||||
Each contributor grants you a non-exclusive, worldwide, royalty-free
|
||||
patent license under the contributor's essential patent claims, to
|
||||
make, use, sell, offer for sale, import and otherwise run, modify and
|
||||
propagate the contents of its contributor version.
|
||||
|
||||
In the following three paragraphs, a "patent license" is any express
|
||||
agreement or commitment, however denominated, not to enforce a patent
|
||||
(such as an express permission to practice a patent or covenant not to
|
||||
sue for patent infringement). To "grant" such a patent license to a
|
||||
party means to make such an agreement or commitment not to enforce a
|
||||
patent against the party.
|
||||
|
||||
If you convey a covered work, knowingly relying on a patent license,
|
||||
and the Corresponding Source of the work is not available for anyone
|
||||
to copy, free of charge and under the terms of this License, through a
|
||||
publicly available network server or other readily accessible means,
|
||||
then you must either (1) cause the Corresponding Source to be so
|
||||
available, or (2) arrange to deprive yourself of the benefit of the
|
||||
patent license for this particular work, or (3) arrange, in a manner
|
||||
consistent with the requirements of this License, to extend the patent
|
||||
license to downstream recipients. "Knowingly relying" means you have
|
||||
actual knowledge that, but for the patent license, your conveying the
|
||||
covered work in a country, or your recipient's use of the covered work
|
||||
in a country, would infringe one or more identifiable patents in that
|
||||
country that you have reason to believe are valid.
|
||||
|
||||
If, pursuant to or in connection with a single transaction or
|
||||
arrangement, you convey, or propagate by procuring conveyance of, a
|
||||
covered work, and grant a patent license to some of the parties
|
||||
receiving the covered work authorizing them to use, propagate, modify
|
||||
or convey a specific copy of the covered work, then the patent license
|
||||
you grant is automatically extended to all recipients of the covered
|
||||
work and works based on it.
|
||||
|
||||
A patent license is "discriminatory" if it does not include within
|
||||
the scope of its coverage, prohibits the exercise of, or is
|
||||
conditioned on the non-exercise of one or more of the rights that are
|
||||
specifically granted under this License. You may not convey a covered
|
||||
work if you are a party to an arrangement with a third party that is
|
||||
in the business of distributing software, under which you make payment
|
||||
to the third party based on the extent of your activity of conveying
|
||||
the work, and under which the third party grants, to any of the
|
||||
parties who would receive the covered work from you, a discriminatory
|
||||
patent license (a) in connection with copies of the covered work
|
||||
conveyed by you (or copies made from those copies), or (b) primarily
|
||||
for and in connection with specific products or compilations that
|
||||
contain the covered work, unless you entered into that arrangement,
|
||||
or that patent license was granted, prior to 28 March 2007.
|
||||
|
||||
Nothing in this License shall be construed as excluding or limiting
|
||||
any implied license or other defenses to infringement that may
|
||||
otherwise be available to you under applicable patent law.
|
||||
|
||||
12. No Surrender of Others' Freedom.
|
||||
|
||||
If conditions are imposed on you (whether by court order, agreement or
|
||||
otherwise) that contradict the conditions of this License, they do not
|
||||
excuse you from the conditions of this License. If you cannot convey a
|
||||
covered work so as to satisfy simultaneously your obligations under this
|
||||
License and any other pertinent obligations, then as a consequence you may
|
||||
not convey it at all. For example, if you agree to terms that obligate you
|
||||
to collect a royalty for further conveying from those to whom you convey
|
||||
the Program, the only way you could satisfy both those terms and this
|
||||
License would be to refrain entirely from conveying the Program.
|
||||
|
||||
13. Use with the GNU Affero General Public License.
|
||||
|
||||
Notwithstanding any other provision of this License, you have
|
||||
permission to link or combine any covered work with a work licensed
|
||||
under version 3 of the GNU Affero General Public License into a single
|
||||
combined work, and to convey the resulting work. The terms of this
|
||||
License will continue to apply to the part which is the covered work,
|
||||
but the special requirements of the GNU Affero General Public License,
|
||||
section 13, concerning interaction through a network will apply to the
|
||||
combination as such.
|
||||
|
||||
14. Revised Versions of this License.
|
||||
|
||||
The Free Software Foundation may publish revised and/or new versions of
|
||||
the GNU General Public License from time to time. Such new versions will
|
||||
be similar in spirit to the present version, but may differ in detail to
|
||||
address new problems or concerns.
|
||||
|
||||
Each version is given a distinguishing version number. If the
|
||||
Program specifies that a certain numbered version of the GNU General
|
||||
Public License "or any later version" applies to it, you have the
|
||||
option of following the terms and conditions either of that numbered
|
||||
version or of any later version published by the Free Software
|
||||
Foundation. If the Program does not specify a version number of the
|
||||
GNU General Public License, you may choose any version ever published
|
||||
by the Free Software Foundation.
|
||||
|
||||
If the Program specifies that a proxy can decide which future
|
||||
versions of the GNU General Public License can be used, that proxy's
|
||||
public statement of acceptance of a version permanently authorizes you
|
||||
to choose that version for the Program.
|
||||
|
||||
Later license versions may give you additional or different
|
||||
permissions. However, no additional obligations are imposed on any
|
||||
author or copyright holder as a result of your choosing to follow a
|
||||
later version.
|
||||
|
||||
15. Disclaimer of Warranty.
|
||||
|
||||
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
|
||||
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
|
||||
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
|
||||
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
|
||||
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
||||
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
|
||||
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
|
||||
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
|
||||
|
||||
16. Limitation of Liability.
|
||||
|
||||
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
|
||||
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
|
||||
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
|
||||
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
|
||||
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
|
||||
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
|
||||
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
|
||||
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
|
||||
SUCH DAMAGES.
|
||||
|
||||
17. Interpretation of Sections 15 and 16.
|
||||
|
||||
If the disclaimer of warranty and limitation of liability provided
|
||||
above cannot be given local legal effect according to their terms,
|
||||
reviewing courts shall apply local law that most closely approximates
|
||||
an absolute waiver of all civil liability in connection with the
|
||||
Program, unless a warranty or assumption of liability accompanies a
|
||||
copy of the Program in return for a fee.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
How to Apply These Terms to Your New Programs
|
||||
|
||||
If you develop a new program, and you want it to be of the greatest
|
||||
possible use to the public, the best way to achieve this is to make it
|
||||
free software which everyone can redistribute and change under these terms.
|
||||
|
||||
To do so, attach the following notices to the program. It is safest
|
||||
to attach them to the start of each source file to most effectively
|
||||
state the exclusion of warranty; and each file should have at least
|
||||
the "copyright" line and a pointer to where the full notice is found.
|
||||
|
||||
<one line to give the program's name and a brief idea of what it does.>
|
||||
Copyright (C) <year> <name of author>
|
||||
|
||||
This program is free software: you can redistribute it and/or modify
|
||||
it under the terms of the GNU General Public License as published by
|
||||
the Free Software Foundation, either version 3 of the License, or
|
||||
(at your option) any later version.
|
||||
|
||||
This program is distributed in the hope that it will be useful,
|
||||
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
GNU General Public License for more details.
|
||||
|
||||
You should have received a copy of the GNU General Public License
|
||||
along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
|
||||
Also add information on how to contact you by electronic and paper mail.
|
||||
|
||||
If the program does terminal interaction, make it output a short
|
||||
notice like this when it starts in an interactive mode:
|
||||
|
||||
<program> Copyright (C) <year> <name of author>
|
||||
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
|
||||
This is free software, and you are welcome to redistribute it
|
||||
under certain conditions; type `show c' for details.
|
||||
|
||||
The hypothetical commands `show w' and `show c' should show the appropriate
|
||||
parts of the General Public License. Of course, your program's commands
|
||||
might be different; for a GUI interface, you would use an "about box".
|
||||
|
||||
You should also get your employer (if you work as a programmer) or school,
|
||||
if any, to sign a "copyright disclaimer" for the program, if necessary.
|
||||
For more information on this, and how to apply and follow the GNU GPL, see
|
||||
<http://www.gnu.org/licenses/>.
|
||||
|
||||
The GNU General Public License does not permit incorporating your program
|
||||
into proprietary programs. If your program is a subroutine library, you
|
||||
may consider it more useful to permit linking proprietary applications with
|
||||
the library. If this is what you want to do, use the GNU Lesser General
|
||||
Public License instead of this License. But first, please read
|
||||
<http://www.gnu.org/philosophy/why-not-lgpl.html>.
|
||||
|
||||
|
|
@ -0,0 +1,56 @@
|
|||
#!/usr/bin/make -f
|
||||
# -*- makefile -*-
|
||||
# Copyright 2011 (C) Daniel Richman. License: GNU GPL 3; see LICENSE.
|
||||
|
||||
CFLAGS = -pthread -O2 -Wall -Werror -pedantic -Wno-variadic-macros -Isrc
|
||||
upl_libs = -pthread -ljsoncpp -lcurl -lssl
|
||||
ext_libs = -ljsoncpp
|
||||
|
||||
test_py_files = tests/test_uploader.py tests/test_extractor.py
|
||||
headers = src/CouchDB.h src/EZ.h src/Uploader.h src/UploaderThread.h \
|
||||
src/Extractor.h src/UKHASExtractor.h \
|
||||
tests/test_extractor_mocks.h
|
||||
upl_cxxfiles = src/CouchDB.cxx src/EZ.cxx src/Uploader.cxx
|
||||
upl_thr_cflags = -DTHREADED
|
||||
upl_nrm_binary = tests/cpp_connector
|
||||
upl_nrm_objects = tests/test_uploader_main.o
|
||||
upl_thr_binary = tests/cpp_connector_threaded
|
||||
upl_thr_objects = src/UploaderThread.o tests/test_uploader_main.threaded.o
|
||||
ext_cxxfiles = src/Extractor.cxx src/UKHASExtractor.cxx \
|
||||
tests/test_extractor_main.cxx
|
||||
ext_binary = tests/extractor
|
||||
ext_mock_cflags = -include tests/test_extractor_mocks.h
|
||||
|
||||
CXXFLAGS = $(CFLAGS)
|
||||
upl_objects = $(patsubst %.cxx,%.o,$(upl_cxxfiles))
|
||||
ext_objects = $(patsubst %.cxx,%.ext_mock.o,$(ext_cxxfiles))
|
||||
|
||||
%.o : %.cxx $(headers)
|
||||
g++ -c $(CXXFLAGS) -o $@ $<
|
||||
|
||||
%.threaded.o : %.cxx $(headers)
|
||||
g++ -c $(CXXFLAGS) $(upl_thr_cflags) -o $@ $<
|
||||
|
||||
%.ext_mock.o : %.cxx $(headers)
|
||||
g++ -c $(CXXFLAGS) $(ext_mock_cflags) -o $@ $<
|
||||
|
||||
$(upl_nrm_binary) : $(upl_objects) $(upl_nrm_objects)
|
||||
g++ $(CXXFLAGS) -o $@ $(upl_objects) $(upl_nrm_objects) $(upl_libs)
|
||||
|
||||
$(upl_thr_binary) : $(upl_objects) $(upl_thr_objects)
|
||||
g++ $(CXXFLAGS) -o $@ $(upl_objects) $(upl_thr_objects) $(upl_libs)
|
||||
|
||||
$(ext_binary) : $(ext_objects)
|
||||
g++ $(CXXFLAGS) -o $@ $(ext_objects) $(ext_libs)
|
||||
|
||||
test : $(upl_nrm_binary) $(upl_thr_binary) $(ext_binary) $(test_py_files)
|
||||
nosetests
|
||||
|
||||
clean :
|
||||
rm -f $(upl_objects) $(upl_nrm_objects) $(upl_thr_objects) \
|
||||
$(upl_nrm_binary) $(upl_thr_binary) \
|
||||
$(ext_objects) $(ext_binary) \
|
||||
$(patsubst %.py,%.pyc,$(test_py_files))
|
||||
|
||||
.PHONY : clean test
|
||||
.DEFAULT_GOAL := test
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
habitat cpp uploader (see habitat.uploader.Uploader)
|
||||
====================================================
|
||||
|
||||
Building dependencies
|
||||
---------------------
|
||||
|
||||
You will need these dependencies:
|
||||
|
||||
- [JsonCpp](http://jsoncpp.sourceforge.net/)
|
||||
- [libcURL](http://curl.haxx.se/)
|
||||
|
||||
Both build from source fairly easily, but the easiest way to acquire them for
|
||||
Ubuntu lucid is:
|
||||
|
||||
- [JsonCpp (PPA)](https://launchpad.net/~danieljonathanrichman/+archive/ppa)
|
||||
- [libcURL (apt)](http://packages.ubuntu.com/lucid/libcurl4-openssl-dev)
|
||||
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
/* Copyright 2011 (C) Daniel Richman; GNU GPL 3 */
|
||||
|
||||
function (doc) {
|
||||
if (doc.type == "flight")
|
||||
emit(doc.end, null);
|
||||
}
|
||||
|
|
@ -0,0 +1,198 @@
|
|||
/* Copyright 2011 (C) Daniel Richman. License: GNU GPL 3; see LICENSE. */
|
||||
|
||||
#include "CouchDB.h"
|
||||
#include <curl/curl.h>
|
||||
#include <string>
|
||||
#include <memory>
|
||||
#include <stdexcept>
|
||||
#include "EZ.h"
|
||||
|
||||
using namespace std;
|
||||
|
||||
namespace CouchDB {
|
||||
|
||||
const map<string,string> Database::view_default_options;
|
||||
|
||||
static string server_url(const string &url)
|
||||
{
|
||||
if (!url.length())
|
||||
throw invalid_argument("URL of zero length");
|
||||
|
||||
string url_ts(url);
|
||||
|
||||
if (*(url.rbegin()) != '/')
|
||||
url_ts.append("/");
|
||||
|
||||
return url_ts;
|
||||
}
|
||||
|
||||
static string database_url(const string &server_url, const string &db)
|
||||
{
|
||||
if (!db.length())
|
||||
throw invalid_argument("DB of zero length");
|
||||
|
||||
string url(server_url);
|
||||
url.append(EZ::cURL::escape(db));
|
||||
|
||||
if (*(db.rbegin()) != '/')
|
||||
url.append("/");
|
||||
|
||||
return url;
|
||||
}
|
||||
|
||||
Server::Server(const string &url)
|
||||
: url(server_url(url)) {}
|
||||
|
||||
Database::Database(Server &server, const string &db)
|
||||
: server(server), url(database_url(server.url, db)) {}
|
||||
|
||||
string Server::next_uuid()
|
||||
{
|
||||
EZ::MutexLock lock(uuid_cache_mutex);
|
||||
string uuid;
|
||||
|
||||
if (uuid_cache.size())
|
||||
{
|
||||
uuid = uuid_cache.front();
|
||||
uuid_cache.pop_front();
|
||||
return uuid;
|
||||
}
|
||||
else
|
||||
{
|
||||
string uuid_url(url);
|
||||
uuid_url.append("_uuids?count=100");
|
||||
|
||||
Json::Value *root = get_json(uuid_url);
|
||||
auto_ptr<Json::Value> value_destroyer(root);
|
||||
|
||||
const Json::Value &uuids = (*root)["uuids"];
|
||||
if (!uuids.isArray() || !uuids.size())
|
||||
throw runtime_error("Invalid UUIDs response");
|
||||
|
||||
uuid = uuids[Json::UInt(0)].asString();
|
||||
|
||||
for (Json::UInt index = 1; index < uuids.size(); index++)
|
||||
uuid_cache.push_back(uuids[index].asString());
|
||||
}
|
||||
|
||||
return uuid;
|
||||
}
|
||||
|
||||
Json::Value *Server::get_json(const string &get_url)
|
||||
{
|
||||
Json::Reader reader;
|
||||
Json::Value *doc = new Json::Value;
|
||||
auto_ptr<Json::Value> value_destroyer(doc);
|
||||
|
||||
string response = curl.get(get_url);
|
||||
|
||||
if (!reader.parse(response, *doc, false))
|
||||
throw runtime_error("JSON Parsing error");
|
||||
|
||||
value_destroyer.release();
|
||||
|
||||
return doc;
|
||||
}
|
||||
|
||||
string Database::make_doc_url(const string &doc_id) const
|
||||
{
|
||||
string doc_url(url);
|
||||
doc_url.append(EZ::cURL::escape(doc_id));
|
||||
return doc_url;
|
||||
}
|
||||
|
||||
Json::Value *Database::operator[](const string &doc_id)
|
||||
{
|
||||
return get_doc(doc_id);
|
||||
}
|
||||
|
||||
void Database::save_doc(Json::Value &doc)
|
||||
{
|
||||
Json::Value &id = doc["_id"];
|
||||
|
||||
if (id.isNull())
|
||||
id = server.next_uuid();
|
||||
|
||||
if (!id.isString())
|
||||
throw runtime_error("_id must be a string if set");
|
||||
|
||||
string doc_id = id.asString();
|
||||
|
||||
if (doc_id.length() == 0)
|
||||
throw runtime_error("_id cannot be an empty string");
|
||||
if (doc_id[0] == '_')
|
||||
throw runtime_error("_id cannot start with _");
|
||||
|
||||
Json::FastWriter writer;
|
||||
string json_doc = writer.write(doc);
|
||||
|
||||
string doc_url = make_doc_url(doc_id);
|
||||
|
||||
string response;
|
||||
|
||||
try
|
||||
{
|
||||
response = server.curl.put(doc_url, json_doc);
|
||||
}
|
||||
catch (EZ::HTTPResponse e)
|
||||
{
|
||||
/* Catch HTTP 409 Resource Conflict */
|
||||
|
||||
if (e.response_code != 409)
|
||||
throw;
|
||||
|
||||
throw Conflict(doc_id);
|
||||
}
|
||||
|
||||
Json::Reader reader;
|
||||
Json::Value info;
|
||||
|
||||
if (!reader.parse(response, info, false))
|
||||
throw runtime_error("JSON Parsing error");
|
||||
|
||||
const Json::Value &new_id = info["id"];
|
||||
const Json::Value &new_rev = info["rev"];
|
||||
|
||||
if (!new_id.isString() || !new_rev.isString())
|
||||
throw runtime_error("Invalid server response (id, rev !string)");
|
||||
|
||||
if (new_id.asString() != doc_id)
|
||||
throw runtime_error("Server has gone insane (saved wrong _id)");
|
||||
|
||||
doc["_rev"] = new_rev;
|
||||
}
|
||||
|
||||
Json::Value *Database::get_doc(const string &doc_id)
|
||||
{
|
||||
return server.get_json(make_doc_url(doc_id));
|
||||
}
|
||||
|
||||
Json::Value *Database::view(const string &design_doc, const string &view_name,
|
||||
const map<string,string> &options)
|
||||
{
|
||||
string view_url(url);
|
||||
|
||||
if (design_doc.length())
|
||||
{
|
||||
view_url.append("_design/");
|
||||
view_url.append(EZ::cURL::escape(design_doc));
|
||||
view_url.append("/_view/");
|
||||
}
|
||||
|
||||
view_url.append(EZ::cURL::escape(view_name));
|
||||
|
||||
if (options.size())
|
||||
{
|
||||
view_url.append(EZ::cURL::query_string(options, true));
|
||||
}
|
||||
|
||||
return server.get_json(view_url);
|
||||
}
|
||||
|
||||
string Database::json_query_value(Json::Value &value)
|
||||
{
|
||||
Json::FastWriter writer;
|
||||
return writer.write(value);
|
||||
}
|
||||
|
||||
} /* namespace CouchDB */
|
||||
|
|
@ -0,0 +1,74 @@
|
|||
/* Copyright 2011 (C) Daniel Richman. License: GNU GPL 3; see LICENSE. */
|
||||
|
||||
#ifndef HABITAT_COUCHDB_H
|
||||
#define HABITAT_COUCHDB_H
|
||||
|
||||
#include <jsoncpp/json.h>
|
||||
#include <string>
|
||||
#include <iostream>
|
||||
#include <deque>
|
||||
#include <stdexcept>
|
||||
#include <curl/curl.h>
|
||||
|
||||
#include "EZ.h"
|
||||
|
||||
using namespace std;
|
||||
|
||||
namespace CouchDB {
|
||||
|
||||
class Server;
|
||||
|
||||
class Database
|
||||
{
|
||||
static const map<string,string> view_default_options;
|
||||
Server &server;
|
||||
string url;
|
||||
friend class Server;
|
||||
|
||||
string make_doc_url(const string &doc_id) const;
|
||||
|
||||
public:
|
||||
Database(Server &server, const string &db);
|
||||
~Database() {};
|
||||
|
||||
void save_doc(Json::Value &doc);
|
||||
Json::Value *get_doc(const string &doc_id);
|
||||
Json::Value *operator[](const string &doc_id);
|
||||
Json::Value *view(const string &design_doc, const string &view_name,
|
||||
const map<string,string> &options=view_default_options);
|
||||
static string json_query_value(Json::Value &value);
|
||||
};
|
||||
|
||||
class Server
|
||||
{
|
||||
const string url;
|
||||
deque<string> uuid_cache;
|
||||
EZ::Mutex uuid_cache_mutex;
|
||||
EZ::cURL curl;
|
||||
|
||||
string next_uuid();
|
||||
friend class Database;
|
||||
|
||||
Json::Value *get_json(const string &get_url);
|
||||
|
||||
public:
|
||||
Server(const string &url);
|
||||
~Server() {};
|
||||
Database operator[](const string &n) { return Database(*this, n); }
|
||||
};
|
||||
|
||||
class Conflict : public runtime_error
|
||||
{
|
||||
Conflict(const string &doc_id)
|
||||
: runtime_error("CouchDB::Conflict: " + doc_id), doc_id(doc_id) {};
|
||||
|
||||
friend class Database;
|
||||
|
||||
public:
|
||||
const string doc_id;
|
||||
~Conflict() throw() {};
|
||||
};
|
||||
|
||||
} /* namespace CouchDB */
|
||||
|
||||
#endif /* HABITAT_COUCHDB_H */
|
||||
|
|
@ -0,0 +1,364 @@
|
|||
/* Copyright 2011 (C) Daniel Richman. License: GNU GPL 3; see LICENSE. */
|
||||
|
||||
#include "EZ.h"
|
||||
#include <curl/curl.h>
|
||||
#include <pthread.h>
|
||||
#include <string>
|
||||
#include <memory>
|
||||
#include <stdexcept>
|
||||
#include <sstream>
|
||||
|
||||
using namespace std;
|
||||
|
||||
namespace EZ {
|
||||
|
||||
class MutexAttr
|
||||
{
|
||||
public:
|
||||
pthread_mutexattr_t attr;
|
||||
|
||||
MutexAttr();
|
||||
~MutexAttr();
|
||||
};
|
||||
|
||||
class ThreadAttr
|
||||
{
|
||||
public:
|
||||
pthread_attr_t attr;
|
||||
|
||||
ThreadAttr();
|
||||
~ThreadAttr();
|
||||
};
|
||||
|
||||
Mutex::Mutex()
|
||||
{
|
||||
MutexAttr attr;
|
||||
int result;
|
||||
|
||||
result = pthread_mutexattr_settype(&attr.attr, PTHREAD_MUTEX_RECURSIVE);
|
||||
|
||||
if (result != 0)
|
||||
throw runtime_error("Failed to set mutex type");
|
||||
|
||||
result = pthread_mutex_init(&mutex, &attr.attr);
|
||||
|
||||
if (result != 0)
|
||||
throw runtime_error("Failed to create mutex");
|
||||
}
|
||||
|
||||
Mutex::~Mutex()
|
||||
{
|
||||
pthread_mutex_destroy(&mutex);
|
||||
}
|
||||
|
||||
MutexLock::MutexLock(Mutex &_m) : m(_m)
|
||||
{
|
||||
pthread_mutex_lock(&(m.mutex));
|
||||
}
|
||||
|
||||
MutexLock::~MutexLock()
|
||||
{
|
||||
pthread_mutex_unlock(&(m.mutex));
|
||||
}
|
||||
|
||||
ConditionVariable::ConditionVariable() : Mutex()
|
||||
{
|
||||
int result = pthread_cond_init(&condvar, NULL);
|
||||
|
||||
if (result != 0)
|
||||
throw runtime_error("Failed to create condvar");
|
||||
}
|
||||
|
||||
ConditionVariable::~ConditionVariable()
|
||||
{
|
||||
pthread_cond_destroy(&condvar);
|
||||
}
|
||||
|
||||
void ConditionVariable::wait()
|
||||
{
|
||||
pthread_cond_wait(&condvar, &mutex);
|
||||
}
|
||||
|
||||
void ConditionVariable::timedwait(const struct timespec *abstime)
|
||||
{
|
||||
pthread_cond_timedwait(&condvar, &mutex, abstime);
|
||||
}
|
||||
|
||||
void ConditionVariable::signal()
|
||||
{
|
||||
pthread_cond_signal(&condvar);
|
||||
}
|
||||
|
||||
void ConditionVariable::broadcast()
|
||||
{
|
||||
pthread_cond_broadcast(&condvar);
|
||||
}
|
||||
|
||||
/* Queue's methods are in UploadThread.h since it's templated */
|
||||
|
||||
MutexAttr::MutexAttr()
|
||||
{
|
||||
int result = pthread_mutexattr_init(&attr);
|
||||
|
||||
if (result != 0)
|
||||
throw runtime_error("Failed to create mutexattr");
|
||||
}
|
||||
|
||||
MutexAttr::~MutexAttr()
|
||||
{
|
||||
pthread_mutexattr_destroy(&attr);
|
||||
}
|
||||
|
||||
ThreadAttr::ThreadAttr()
|
||||
{
|
||||
int result = pthread_attr_init(&attr);
|
||||
|
||||
if (result != 0)
|
||||
throw runtime_error("Failed to create attr");
|
||||
}
|
||||
|
||||
ThreadAttr::~ThreadAttr()
|
||||
{
|
||||
pthread_attr_destroy(&attr);
|
||||
}
|
||||
|
||||
SimpleThread::SimpleThread()
|
||||
: started(false), joined(false), exit_arg(NULL)
|
||||
{}
|
||||
|
||||
SimpleThread::~SimpleThread()
|
||||
{
|
||||
MutexLock lock(mutex);
|
||||
|
||||
if (started && !joined)
|
||||
join();
|
||||
}
|
||||
|
||||
void *thread_starter(void *arg)
|
||||
{
|
||||
SimpleThread *t = static_cast<SimpleThread *>(arg);
|
||||
return t->run();
|
||||
}
|
||||
|
||||
void SimpleThread::start()
|
||||
{
|
||||
MutexLock lock(mutex);
|
||||
|
||||
if (started)
|
||||
return;
|
||||
|
||||
ThreadAttr attr;
|
||||
int result;
|
||||
|
||||
result = pthread_attr_setdetachstate(&attr.attr, PTHREAD_CREATE_JOINABLE);
|
||||
|
||||
if (result != 0)
|
||||
throw runtime_error("Failed to set detach state");
|
||||
|
||||
result = pthread_create(&thread, &attr.attr, thread_starter, this);
|
||||
|
||||
if (result != 0)
|
||||
throw runtime_error("Failed to create a thread");
|
||||
|
||||
started = true;
|
||||
}
|
||||
|
||||
void *SimpleThread::join()
|
||||
{
|
||||
MutexLock lock(mutex);
|
||||
|
||||
if (!started)
|
||||
throw runtime_error("Cannot join a thread that hasn't started");
|
||||
|
||||
if (joined)
|
||||
return NULL;
|
||||
|
||||
int result = pthread_join(thread, &exit_arg);
|
||||
if (result != 0)
|
||||
throw runtime_error("Failed to join thread");
|
||||
|
||||
joined = true;
|
||||
return exit_arg;
|
||||
}
|
||||
|
||||
static string http_response_string(long r, string u)
|
||||
{
|
||||
stringstream ss;
|
||||
ss << "EZ::HTTPResponse: HTTP " << r << " (" << u << ")";
|
||||
return ss.str();
|
||||
}
|
||||
|
||||
HTTPResponse::HTTPResponse(long r, string u)
|
||||
: runtime_error(http_response_string(r, u)), response_code(r), url(u) {}
|
||||
|
||||
cURL::cURL()
|
||||
{
|
||||
curl = curl_easy_init();
|
||||
|
||||
if (curl == NULL)
|
||||
{
|
||||
throw runtime_error("Failed to create curl");
|
||||
}
|
||||
}
|
||||
|
||||
cURL::~cURL()
|
||||
{
|
||||
curl_easy_cleanup(curl);
|
||||
}
|
||||
|
||||
string cURL::escape(const string &s)
|
||||
{
|
||||
char *result;
|
||||
|
||||
/* cURL wants a handle passed to easy escape for some reason.
|
||||
* As far as I can tell it doesn't use it... */
|
||||
cURL escaper;
|
||||
MutexLock lock(escaper.mutex);
|
||||
|
||||
result = curl_easy_escape(escaper.curl, s.c_str(), s.length());
|
||||
|
||||
if (result == NULL)
|
||||
throw runtime_error("curl_easy_escape failed");
|
||||
|
||||
string result_string(result);
|
||||
curl_free(result);
|
||||
|
||||
return result_string;
|
||||
}
|
||||
|
||||
string cURL::query_string(const map<string,string> &options,
|
||||
bool add_questionmark)
|
||||
{
|
||||
string result;
|
||||
|
||||
if (add_questionmark)
|
||||
result.append("?");
|
||||
|
||||
map<string,string>::const_iterator it;
|
||||
|
||||
for (it = options.begin(); it != options.end(); it++)
|
||||
{
|
||||
if (it != options.begin())
|
||||
result.append("&");
|
||||
|
||||
string key_escaped = escape((*it).first);
|
||||
string value_escaped = escape((*it).second);
|
||||
|
||||
result.append(key_escaped);
|
||||
result.append("=");
|
||||
result.append(value_escaped);
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
void cURL::reset()
|
||||
{
|
||||
curl_easy_reset(curl);
|
||||
}
|
||||
|
||||
template<typename T> void cURL::setopt(CURLoption option, T parameter)
|
||||
{
|
||||
CURLcode result = curl_easy_setopt(curl, option, parameter);
|
||||
if (result != CURLE_OK)
|
||||
throw cURLError(result, "curl_easy_setopt");
|
||||
}
|
||||
|
||||
string cURL::get(const string &url)
|
||||
{
|
||||
MutexLock lock(mutex);
|
||||
|
||||
reset();
|
||||
return cURL::perform(url);
|
||||
}
|
||||
|
||||
string cURL::post(const string &url, const string &data)
|
||||
{
|
||||
MutexLock lock(mutex);
|
||||
|
||||
reset();
|
||||
setopt(CURLOPT_POSTFIELDS, data.c_str());
|
||||
setopt(CURLOPT_POSTFIELDSIZE, data.length());
|
||||
|
||||
return cURL::perform(url);
|
||||
}
|
||||
|
||||
struct read_func_userdata
|
||||
{
|
||||
const string *data;
|
||||
size_t written;
|
||||
};
|
||||
|
||||
static size_t read_func(void *ptr, size_t size, size_t nmemb, void *userdata)
|
||||
{
|
||||
struct read_func_userdata *source =
|
||||
static_cast<struct read_func_userdata *>(userdata);
|
||||
char *target = static_cast<char *>(ptr);
|
||||
size_t max = size * nmemb;
|
||||
size_t remaining = source->data->length() - source->written;
|
||||
|
||||
size_t write = remaining;
|
||||
if (write > max)
|
||||
write = max;
|
||||
|
||||
if (write)
|
||||
{
|
||||
source->data->copy(target, write, source->written);
|
||||
source->written += write;
|
||||
}
|
||||
|
||||
return write;
|
||||
}
|
||||
|
||||
string cURL::put(const string &url, const string &data)
|
||||
{
|
||||
MutexLock lock(mutex);
|
||||
|
||||
reset();
|
||||
setopt(CURLOPT_UPLOAD, 1);
|
||||
|
||||
struct read_func_userdata userdata;
|
||||
userdata.data = &data;
|
||||
userdata.written = 0;
|
||||
|
||||
setopt(CURLOPT_READFUNCTION, read_func);
|
||||
setopt(CURLOPT_READDATA, &userdata);
|
||||
setopt(CURLOPT_INFILESIZE, data.length());
|
||||
|
||||
return cURL::perform(url);
|
||||
}
|
||||
|
||||
static size_t write_func(char *data, size_t size, size_t nmemb, void *userdata)
|
||||
{
|
||||
size_t length = size * nmemb;
|
||||
string *target = static_cast<string *>(userdata);
|
||||
|
||||
target->append(data, length);
|
||||
return length;
|
||||
}
|
||||
|
||||
string cURL::perform(const string &url)
|
||||
{
|
||||
string response;
|
||||
|
||||
setopt(CURLOPT_URL, url.c_str());
|
||||
setopt(CURLOPT_WRITEFUNCTION, write_func);
|
||||
setopt(CURLOPT_WRITEDATA, &response);
|
||||
|
||||
CURLcode result;
|
||||
result = curl_easy_perform(curl);
|
||||
if (result != CURLE_OK)
|
||||
throw cURLError(result, "curl_easy_perform");
|
||||
|
||||
long response_code;
|
||||
result = curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &response_code);
|
||||
if (result != CURLE_OK)
|
||||
throw cURLError(result, "curl_easy_getinfo");
|
||||
|
||||
if (response_code < 200 || response_code > 299)
|
||||
throw HTTPResponse(response_code, url);
|
||||
|
||||
return response;
|
||||
}
|
||||
|
||||
} /* namespace EZ */
|
||||
|
|
@ -0,0 +1,180 @@
|
|||
/* Copyright 2011 (C) Daniel Richman. License: GNU GPL 3; see LICENSE. */
|
||||
|
||||
#ifndef HABITAT_EZ_H
|
||||
#define HABITAT_EZ_H
|
||||
|
||||
#include <string>
|
||||
#include <iostream>
|
||||
#include <stdexcept>
|
||||
#include <map>
|
||||
#include <deque>
|
||||
#include <curl/curl.h>
|
||||
#include <pthread.h>
|
||||
|
||||
using namespace std;
|
||||
|
||||
namespace EZ {
|
||||
|
||||
class Mutex
|
||||
{
|
||||
protected:
|
||||
pthread_mutex_t mutex;
|
||||
|
||||
friend class MutexLock;
|
||||
|
||||
public:
|
||||
Mutex();
|
||||
~Mutex();
|
||||
};
|
||||
|
||||
class MutexLock
|
||||
{
|
||||
Mutex &m;
|
||||
|
||||
public:
|
||||
MutexLock(Mutex &_m);
|
||||
~MutexLock();
|
||||
};
|
||||
|
||||
class ConditionVariable : public Mutex
|
||||
{
|
||||
pthread_cond_t condvar;
|
||||
|
||||
public:
|
||||
ConditionVariable();
|
||||
~ConditionVariable();
|
||||
|
||||
/* You *need* to have the mutex to do this!
|
||||
* Create an EZ::MutexLock on the ConditionVariable */
|
||||
void wait();
|
||||
void timedwait(const struct timespec *abstime);
|
||||
void signal();
|
||||
void broadcast();
|
||||
};
|
||||
|
||||
template <typename item>
|
||||
class Queue
|
||||
{
|
||||
ConditionVariable condvar;
|
||||
deque<item> item_deque;
|
||||
|
||||
public:
|
||||
void put(item &x);
|
||||
item get();
|
||||
};
|
||||
|
||||
template <typename item>
|
||||
void Queue<item>::put(item &x)
|
||||
{
|
||||
MutexLock lock(condvar);
|
||||
item_deque.push_back(x);
|
||||
condvar.signal();
|
||||
}
|
||||
|
||||
template <typename item>
|
||||
item Queue<item>::get()
|
||||
{
|
||||
MutexLock lock(condvar);
|
||||
|
||||
while (!item_deque.size())
|
||||
condvar.wait();
|
||||
|
||||
item x = item_deque.front();
|
||||
item_deque.pop_front();
|
||||
return x;
|
||||
}
|
||||
|
||||
class SimpleThread
|
||||
{
|
||||
protected:
|
||||
Mutex mutex;
|
||||
pthread_t thread;
|
||||
|
||||
private:
|
||||
bool started;
|
||||
bool joined;
|
||||
void *exit_arg;
|
||||
|
||||
public:
|
||||
SimpleThread();
|
||||
virtual ~SimpleThread();
|
||||
|
||||
virtual void *run() = 0;
|
||||
void start();
|
||||
void *join();
|
||||
};
|
||||
|
||||
class cURL
|
||||
{
|
||||
Mutex mutex;
|
||||
CURL *curl;
|
||||
|
||||
/* You need to hold the mutex to use these four functions */
|
||||
void reset();
|
||||
string perform(const string &url);
|
||||
template<typename T> void setopt(CURLoption option, T paramater);
|
||||
void setopt(CURLoption option, void *paramater);
|
||||
void setopt(CURLoption option, long parameter);
|
||||
|
||||
public:
|
||||
cURL();
|
||||
~cURL();
|
||||
static string escape(const string &s);
|
||||
static string query_string(const map<string,string> &options,
|
||||
bool add_questionmark=false);
|
||||
string get(const string &url);
|
||||
string post(const string &url, const string &data);
|
||||
string put(const string &url, const string &data);
|
||||
};
|
||||
|
||||
class cURLGlobal
|
||||
{
|
||||
public:
|
||||
cURLGlobal() { curl_global_init(CURL_GLOBAL_ALL); };
|
||||
~cURLGlobal() { curl_global_cleanup(); };
|
||||
};
|
||||
|
||||
class cURLslist
|
||||
{
|
||||
struct curl_slist *slist;
|
||||
|
||||
public:
|
||||
cURLslist() { slist = NULL; };
|
||||
void append(const char *s) { curl_slist_append(slist, s); };
|
||||
~cURLslist() { curl_slist_free_all(slist); };
|
||||
const struct curl_slist *get() { return slist; };
|
||||
};
|
||||
|
||||
class cURLError : public runtime_error
|
||||
{
|
||||
cURLError(const string &what)
|
||||
: runtime_error("EZ::cURLError: " + what),
|
||||
error(CURLE_OK), function("") {};
|
||||
|
||||
cURLError(CURLcode error, const string &function)
|
||||
: runtime_error("EZ::cURLError: " + function + ": " +
|
||||
curl_easy_strerror(error)),
|
||||
error(error), function(function) {};
|
||||
|
||||
friend class cURL;
|
||||
|
||||
public:
|
||||
const CURLcode error;
|
||||
const string function;
|
||||
~cURLError() throw() {};
|
||||
};
|
||||
|
||||
class HTTPResponse : public runtime_error
|
||||
{
|
||||
HTTPResponse(long r, string u);
|
||||
friend class cURL;
|
||||
|
||||
public:
|
||||
const long response_code;
|
||||
const string url;
|
||||
~HTTPResponse() throw() {};
|
||||
};
|
||||
|
||||
} /* namespace EZ */
|
||||
|
||||
#endif /* HABITAT_EZ_H */
|
||||
|
|
@ -0,0 +1,49 @@
|
|||
/* Copyright 2011 (C) Daniel Richman. License: GNU GPL 3; see COPYING. */
|
||||
|
||||
#include "Extractor.h"
|
||||
#include <vector>
|
||||
#include "EZ.h"
|
||||
|
||||
namespace habitat {
|
||||
|
||||
void ExtractorManager::add(Extractor &e)
|
||||
{
|
||||
EZ::MutexLock lock(mutex);
|
||||
|
||||
extractors.push_back(&e);
|
||||
e.mgr = this;
|
||||
}
|
||||
|
||||
void ExtractorManager::skipped(int n)
|
||||
{
|
||||
EZ::MutexLock lock(mutex);
|
||||
|
||||
vector<Extractor *>::iterator it;
|
||||
|
||||
for (it = extractors.begin(); it != extractors.end(); it++)
|
||||
(*it)->skipped(n);
|
||||
}
|
||||
|
||||
void ExtractorManager::push(char b, enum push_flags flags)
|
||||
{
|
||||
EZ::MutexLock lock(mutex);
|
||||
|
||||
vector<Extractor *>::iterator it;
|
||||
|
||||
for (it = extractors.begin(); it != extractors.end(); it++)
|
||||
(*it)->push(b, flags);
|
||||
}
|
||||
|
||||
void ExtractorManager::payload(const Json::Value *set)
|
||||
{
|
||||
EZ::MutexLock lock(mutex);
|
||||
current_payload = set;
|
||||
}
|
||||
|
||||
const Json::Value *ExtractorManager::payload()
|
||||
{
|
||||
EZ::MutexLock lock(mutex);
|
||||
return current_payload;
|
||||
}
|
||||
|
||||
} /* namespace habitat */
|
||||
|
|
@ -0,0 +1,58 @@
|
|||
/* Copyright 2011 (C) Daniel Richman. License: GNU GPL 3; see COPYING. */
|
||||
|
||||
#ifndef HABITAT_EXTRACTOR_H
|
||||
#define HABITAT_EXTRACTOR_H
|
||||
|
||||
#include <vector>
|
||||
#include <jsoncpp/json.h>
|
||||
#include "UploaderThread.h"
|
||||
#include "EZ.h"
|
||||
|
||||
using namespace std;
|
||||
|
||||
namespace habitat {
|
||||
|
||||
enum push_flags
|
||||
{
|
||||
PUSH_NONE = 0x00,
|
||||
PUSH_BAUDOT_HACK = 0x01
|
||||
};
|
||||
|
||||
class Extractor;
|
||||
|
||||
class ExtractorManager
|
||||
{
|
||||
EZ::Mutex mutex;
|
||||
vector<Extractor *> extractors;
|
||||
const Json::Value *current_payload;
|
||||
|
||||
public:
|
||||
UploaderThread &uthr;
|
||||
|
||||
ExtractorManager(UploaderThread &u) : current_payload(NULL), uthr(u) {};
|
||||
virtual ~ExtractorManager() {};
|
||||
|
||||
void add(Extractor &e);
|
||||
void skipped(int n);
|
||||
void push(char b, enum push_flags flags=PUSH_NONE);
|
||||
void payload(const Json::Value *set);
|
||||
const Json::Value *payload();
|
||||
|
||||
virtual void status(const string &msg) = 0;
|
||||
virtual void data(const Json::Value &d) = 0;
|
||||
};
|
||||
|
||||
class Extractor
|
||||
{
|
||||
protected:
|
||||
ExtractorManager *mgr;
|
||||
friend void ExtractorManager::add(Extractor &e);
|
||||
|
||||
public:
|
||||
virtual void skipped(int n) = 0;
|
||||
virtual void push(char b, enum push_flags flags) = 0;
|
||||
};
|
||||
|
||||
} /* namespace habitat */
|
||||
|
||||
#endif /* HABITAT_EXTRACTOR_H */
|
||||
|
|
@ -0,0 +1,400 @@
|
|||
/* Copyright 2011 (C) Daniel Richman. License: GNU GPL 3; see COPYING. */
|
||||
|
||||
#include "UKHASExtractor.h"
|
||||
#include <jsoncpp/json.h>
|
||||
#include <stdexcept>
|
||||
#include <string>
|
||||
#include <sstream>
|
||||
#include <algorithm>
|
||||
#include <stdio.h>
|
||||
#include <stdint.h>
|
||||
|
||||
using namespace std;
|
||||
|
||||
namespace habitat {
|
||||
|
||||
void UKHASExtractor::reset_buffer()
|
||||
{
|
||||
buffer.resize(0);
|
||||
buffer.clear();
|
||||
buffer.reserve(256);
|
||||
}
|
||||
|
||||
void UKHASExtractor::skipped(int n)
|
||||
{
|
||||
if (n > 20)
|
||||
n = 20;
|
||||
|
||||
for (int i = 0; i < n; i++)
|
||||
push('\0', PUSH_NONE);
|
||||
}
|
||||
|
||||
void UKHASExtractor::push(char b, enum push_flags flags)
|
||||
{
|
||||
if (b == '\r') b = '\n';
|
||||
|
||||
if (last == '$' && b == '$')
|
||||
{
|
||||
/* Start delimiter: "$$" */
|
||||
reset_buffer();
|
||||
buffer.push_back(last);
|
||||
buffer.push_back(b);
|
||||
|
||||
garbage_count = 0;
|
||||
extracting = true;
|
||||
|
||||
mgr->status("UKHAS Extractor: found start delimiter");
|
||||
}
|
||||
else if (extracting && b == '\n')
|
||||
{
|
||||
/* End delimiter: "\n" */
|
||||
buffer.push_back(b);
|
||||
mgr->uthr.payload_telemetry(buffer);
|
||||
|
||||
mgr->status("UKHAS Extractor: extracted string");
|
||||
|
||||
try
|
||||
{
|
||||
mgr->data(crude_parse());
|
||||
}
|
||||
catch (runtime_error e)
|
||||
{
|
||||
mgr->status("UKHAS Extractor: crude parse failed: " +
|
||||
string(e.what()));
|
||||
|
||||
Json::Value bare(Json::objectValue);
|
||||
bare["_sentence"] = buffer;
|
||||
mgr->data(bare);
|
||||
}
|
||||
|
||||
reset_buffer();
|
||||
extracting = false;
|
||||
}
|
||||
else if (extracting)
|
||||
{
|
||||
/* baudot doesn't support '*', so we use '#'. */
|
||||
if ((flags & PUSH_BAUDOT_HACK) && b == '#')
|
||||
b = '*';
|
||||
|
||||
buffer.push_back(b);
|
||||
|
||||
if (b < 0x20 || b > 0x7E)
|
||||
garbage_count++;
|
||||
|
||||
/* Sane limits to avoid uploading tonnes of garbage */
|
||||
if (buffer.length() > 1000 || garbage_count > 16)
|
||||
{
|
||||
mgr->status("UKHAS Extractor: giving up");
|
||||
|
||||
reset_buffer();
|
||||
extracting = false;
|
||||
}
|
||||
}
|
||||
|
||||
last = b;
|
||||
}
|
||||
|
||||
static void inplace_toupper(char &c)
|
||||
{
|
||||
if (c >= 'a' && c <= 'z')
|
||||
c -= 32;
|
||||
}
|
||||
|
||||
static string checksum_xor(const string &s)
|
||||
{
|
||||
uint8_t checksum = 0;
|
||||
for (string::const_iterator it = s.begin(); it != s.end(); it++)
|
||||
checksum ^= (*it);
|
||||
|
||||
char temp[3];
|
||||
snprintf(temp, sizeof(temp), "%.02X", checksum);
|
||||
return string(temp);
|
||||
}
|
||||
|
||||
static string checksum_crc16_ccitt(const string &s)
|
||||
{
|
||||
/* From avr-libc docs: Modified BSD (GPL, BSD, DFSG compatible) */
|
||||
uint16_t crc = 0xFFFF;
|
||||
|
||||
for (string::const_iterator it = s.begin(); it != s.end(); it++)
|
||||
{
|
||||
crc = crc ^ ((uint16_t (*it)) << 8);
|
||||
|
||||
for (int i = 0; i < 8; i++)
|
||||
{
|
||||
bool s = crc & 0x8000;
|
||||
crc <<= 1;
|
||||
crc ^= (s ? 0x1021 : 0);
|
||||
}
|
||||
}
|
||||
|
||||
char temp[5];
|
||||
snprintf(temp, sizeof(temp), "%.04X", crc);
|
||||
return string(temp);
|
||||
}
|
||||
|
||||
static vector<string> split(const string &input, const char c)
|
||||
{
|
||||
vector<string> parts;
|
||||
size_t pos = 0, lastpos = 0;
|
||||
|
||||
do
|
||||
{
|
||||
/* pos returns npos? substr will grab to end of string. */
|
||||
pos = input.find_first_of(c, lastpos);
|
||||
|
||||
if (pos == string::npos)
|
||||
parts.push_back(input.substr(lastpos));
|
||||
else
|
||||
parts.push_back(input.substr(lastpos, pos - lastpos));
|
||||
|
||||
lastpos = pos + 1;
|
||||
}
|
||||
while (pos != string::npos);
|
||||
|
||||
return parts;
|
||||
}
|
||||
|
||||
static void split_string(const string &buffer, string *data, string *checksum)
|
||||
{
|
||||
if (buffer.substr(0, 2) != "$$")
|
||||
throw runtime_error("String does not begin with $$");
|
||||
|
||||
if (buffer[buffer.length() - 1] != '\n')
|
||||
throw runtime_error("String does not end with '\\n'");
|
||||
|
||||
size_t pos = buffer.find_last_of('*');
|
||||
if (pos == string::npos)
|
||||
throw runtime_error("No checksum");
|
||||
|
||||
size_t check_start = pos + 1;
|
||||
size_t check_end = buffer.length() - 1;
|
||||
size_t check_length = check_end - check_start;
|
||||
|
||||
if (check_length != 2 && check_length != 4)
|
||||
throw runtime_error("Invalid checksum length");
|
||||
|
||||
size_t data_start = 2;
|
||||
size_t data_length = pos - data_start;
|
||||
|
||||
*data = buffer.substr(data_start, data_length);
|
||||
*checksum = buffer.substr(check_start, check_length);
|
||||
}
|
||||
|
||||
static string examine_checksum(const string &data, const string &checksum_o)
|
||||
{
|
||||
string checksum = checksum_o;
|
||||
for_each(checksum.begin(), checksum.end(), inplace_toupper);
|
||||
|
||||
string expect, name;
|
||||
|
||||
if (checksum.length() == 2)
|
||||
{
|
||||
expect = checksum_xor(data);
|
||||
name = "xor";
|
||||
}
|
||||
else if (checksum.length() == 4)
|
||||
{
|
||||
expect = checksum_crc16_ccitt(data);
|
||||
name = "crc16-ccitt";
|
||||
}
|
||||
else
|
||||
{
|
||||
throw runtime_error("Invalid checksum length");
|
||||
}
|
||||
|
||||
if (expect != checksum)
|
||||
throw runtime_error("Invalid checksum: expected " + expect);
|
||||
|
||||
return name;
|
||||
}
|
||||
|
||||
static bool is_ddmmmm_field(const Json::Value &field)
|
||||
{
|
||||
if (field["sensor"] != "stdtelem.coordinate")
|
||||
return false;
|
||||
|
||||
if (!field["format"].isString())
|
||||
return false;
|
||||
|
||||
string format = field["format"].asString();
|
||||
|
||||
/* does it match d+m+\.m+ ? */
|
||||
|
||||
size_t pos;
|
||||
|
||||
pos = format.find_first_not_of('d');
|
||||
if (pos == string::npos || format[pos] != 'm')
|
||||
return false;
|
||||
|
||||
pos = format.find_first_not_of('m', pos);
|
||||
if (pos == string::npos || format[pos] != '.')
|
||||
return false;
|
||||
|
||||
pos++;
|
||||
|
||||
pos = format.find_first_not_of('m', pos);
|
||||
if (pos != string::npos)
|
||||
return false;
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
static string convert_ddmmmm(const string &value)
|
||||
{
|
||||
size_t split = value.find('.');
|
||||
if (split == string::npos || split <= 2)
|
||||
throw runtime_error("invalid '.' pos when converting ddmm");
|
||||
|
||||
string left = value.substr(0, split - 2);
|
||||
string right = value.substr(split - 2);
|
||||
|
||||
istringstream lis(left), ris(right);
|
||||
double left_val, right_val;
|
||||
lis >> left_val;
|
||||
ris >> right_val;
|
||||
|
||||
if (lis.fail() || ris.fail() || lis.peek() != EOF || ris.peek() != EOF)
|
||||
throw runtime_error("couldn't parse left or right parts (ddmm)");
|
||||
|
||||
if (right_val >= 60 || right_val < 0)
|
||||
throw runtime_error("invalid right part (ddmm)");
|
||||
|
||||
double dd = left_val + (right_val / 60);
|
||||
|
||||
ostringstream os;
|
||||
os.precision(value.length() - value.find_first_not_of("0+-") - 2);
|
||||
os << dd;
|
||||
return os.str();
|
||||
}
|
||||
|
||||
static void extract_fields(Json::Value &data, const Json::Value &fields,
|
||||
const vector<string> &parts)
|
||||
{
|
||||
vector<string>::const_iterator part = parts.begin() + 1;
|
||||
Json::Value::const_iterator field = fields.begin();
|
||||
|
||||
while (field != fields.end() && part != parts.end())
|
||||
{
|
||||
const string key = (*field)["name"].asString();
|
||||
const string value = (*part);
|
||||
|
||||
if (!key.length())
|
||||
throw runtime_error("Invalid configuration (empty field name)");
|
||||
|
||||
if (value.length())
|
||||
{
|
||||
if (is_ddmmmm_field(*field))
|
||||
data[key] = convert_ddmmmm(value);
|
||||
else
|
||||
data[key] = value;
|
||||
}
|
||||
|
||||
field++;
|
||||
part++;
|
||||
}
|
||||
}
|
||||
|
||||
static void cook_basic(Json::Value &basic, const string &buffer,
|
||||
const string &callsign)
|
||||
{
|
||||
basic["_sentence"] = buffer;
|
||||
basic["_protocol"] = "UKHAS";
|
||||
basic["_parsed"] = true;
|
||||
basic["payload"] = callsign;
|
||||
}
|
||||
|
||||
static void attempt_settings(Json::Value &data, const Json::Value &sentence,
|
||||
const string &checksum_name,
|
||||
const vector<string> &parts)
|
||||
{
|
||||
const Json::Value &fields = sentence["fields"];
|
||||
|
||||
const string callsign = sentence["payload"].asString();
|
||||
if (parts[0] != callsign)
|
||||
throw runtime_error("Incorrect callsign");
|
||||
|
||||
if (sentence["checksum"] != checksum_name)
|
||||
throw runtime_error("Wrong checksum type");
|
||||
|
||||
if (fields.size() != (parts.size() - 1))
|
||||
throw runtime_error("Incorrect number of fields");
|
||||
|
||||
extract_fields(data, fields, parts);
|
||||
}
|
||||
|
||||
/* crude_parse is based on the parse() method of
|
||||
* habitat.parser_modules.ukhas_parser.UKHASParser */
|
||||
Json::Value UKHASExtractor::crude_parse()
|
||||
{
|
||||
const Json::Value *settings_ptr = mgr->payload();
|
||||
|
||||
if (!settings_ptr)
|
||||
settings_ptr = &(Json::Value::null);
|
||||
|
||||
const Json::Value &settings = *settings_ptr;
|
||||
|
||||
string data, checksum;
|
||||
split_string(buffer, &data, &checksum);
|
||||
|
||||
/* Warning: cpp_connector only supports xor and crc16-ccitt, which
|
||||
* conveninently are different lengths, so this works. */
|
||||
string checksum_name = examine_checksum(data, checksum);
|
||||
vector<string> parts = split(data, ',');
|
||||
if (!parts.size() || !parts[0].size())
|
||||
throw runtime_error("Empty callsign");
|
||||
|
||||
Json::Value basic(Json::objectValue);
|
||||
cook_basic(basic, buffer, parts[0]);
|
||||
const Json::Value &sentence = settings["sentence"];
|
||||
|
||||
/* If array: multiple sentence settings to try with.
|
||||
* No settings? No problem; we can still test the checksum */
|
||||
if (!sentence.isNull() && sentence.isArray())
|
||||
{
|
||||
/* Silence errors, and only log them if all attempts fail */
|
||||
vector<string> errors;
|
||||
|
||||
for (Json::Value::iterator it = sentence.begin();
|
||||
it != sentence.end(); it++)
|
||||
{
|
||||
try
|
||||
{
|
||||
Json::Value data(basic);
|
||||
attempt_settings(data, (*it), checksum_name, parts);
|
||||
return data;
|
||||
}
|
||||
catch (runtime_error e)
|
||||
{
|
||||
errors.push_back(e.what());
|
||||
}
|
||||
}
|
||||
|
||||
/* Couldn't parse using any of the settings... */
|
||||
mgr->status("UKHAS Extractor: full parse failed:");
|
||||
for (vector<string>::iterator it = errors.begin();
|
||||
it != errors.end(); it++)
|
||||
{
|
||||
mgr->status("UKHAS Extractor: " + (*it));
|
||||
}
|
||||
}
|
||||
else if (!sentence.isNull() && sentence.isObject())
|
||||
{
|
||||
try
|
||||
{
|
||||
Json::Value data(basic);
|
||||
attempt_settings(data, sentence, checksum_name, parts);
|
||||
return data;
|
||||
}
|
||||
catch (runtime_error e)
|
||||
{
|
||||
mgr->status("UKHAS Extractor: full parse failed: " +
|
||||
string(e.what()));
|
||||
}
|
||||
}
|
||||
|
||||
basic["_basic"] = true;
|
||||
return basic;
|
||||
}
|
||||
|
||||
} /* namespace habitat */
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
/* Copyright 2011 (C) Daniel Richman. License: GNU GPL 3; see COPYING. */
|
||||
|
||||
#ifndef HABITAT_UKHAS_EXTRACTOR_H
|
||||
#define HABITAT_UKHAS_EXTRACTOR_H
|
||||
|
||||
#include "Extractor.h"
|
||||
|
||||
namespace habitat {
|
||||
|
||||
class UKHASExtractor : public Extractor
|
||||
{
|
||||
int extracting;
|
||||
char last;
|
||||
string buffer;
|
||||
int garbage_count;
|
||||
|
||||
void reset_buffer();
|
||||
Json::Value crude_parse();
|
||||
|
||||
public:
|
||||
UKHASExtractor() : extracting(false), last('\0'), garbage_count(0) {};
|
||||
~UKHASExtractor() {};
|
||||
void skipped(int n);
|
||||
void push(char b, enum push_flags flags);
|
||||
};
|
||||
|
||||
} /* namespace habitat */
|
||||
|
||||
#endif /* HABITAT_UKHAS_EXTRACTOR_H */
|
||||
|
|
@ -0,0 +1,276 @@
|
|||
/* Copyright 2011 (C) Daniel Richman. License: GNU GPL 3; see LICENSE. */
|
||||
|
||||
#include "Uploader.h"
|
||||
#include <string>
|
||||
#include <vector>
|
||||
#include <memory>
|
||||
#include <sstream>
|
||||
#include <stdexcept>
|
||||
#include <openssl/sha.h>
|
||||
#include <openssl/bio.h>
|
||||
#include <openssl/evp.h>
|
||||
#include "CouchDB.h"
|
||||
#include "EZ.h"
|
||||
|
||||
using namespace std;
|
||||
|
||||
namespace habitat {
|
||||
|
||||
Uploader::Uploader(const string &callsign, const string &couch_uri,
|
||||
const string &couch_db, int max_merge_attempts)
|
||||
: callsign(callsign), server(couch_uri), database(server, couch_db),
|
||||
max_merge_attempts(max_merge_attempts)
|
||||
{
|
||||
if (!callsign.length())
|
||||
throw invalid_argument("Callsign of zero length");
|
||||
}
|
||||
|
||||
static char hexchar(int n)
|
||||
{
|
||||
if (n < 10)
|
||||
return '0' + n;
|
||||
else
|
||||
return 'a' + n - 10;
|
||||
}
|
||||
|
||||
static string sha256hex(const string &data)
|
||||
{
|
||||
unsigned char hash[SHA256_DIGEST_LENGTH];
|
||||
string hexhash;
|
||||
hexhash.reserve(SHA256_DIGEST_LENGTH * 2);
|
||||
|
||||
const unsigned char *dc =
|
||||
reinterpret_cast<const unsigned char *>(data.c_str());
|
||||
SHA256(dc, data.length(), hash);
|
||||
|
||||
for (int i = 0; i < SHA256_DIGEST_LENGTH; i++)
|
||||
{
|
||||
char tmp[2];
|
||||
tmp[0] = hexchar((hash[i] & 0xF0) >> 4);
|
||||
tmp[1] = hexchar(hash[i] & 0x0F);
|
||||
hexhash.append(tmp, 2);
|
||||
}
|
||||
|
||||
return hexhash;
|
||||
}
|
||||
|
||||
static string base64(const string &data)
|
||||
{
|
||||
/* So it's either this or linking with another b64 library... */
|
||||
BIO *bio_mem, *bio_b64;
|
||||
|
||||
bio_b64 = BIO_new(BIO_f_base64());
|
||||
bio_mem = BIO_new(BIO_s_mem());
|
||||
if (bio_b64 == NULL || bio_mem == NULL)
|
||||
throw runtime_error("Base64 conversion failed");
|
||||
|
||||
BIO_set_flags(bio_b64, BIO_FLAGS_BASE64_NO_NL);
|
||||
|
||||
bio_b64 = BIO_push(bio_b64, bio_mem);
|
||||
/* Chain is now ->b64->mem */
|
||||
|
||||
size_t result_a;
|
||||
int result_b;
|
||||
|
||||
result_a = BIO_write(bio_b64, data.c_str(), data.length());
|
||||
result_b = BIO_flush(bio_b64);
|
||||
|
||||
if (result_a != data.length() || result_b != 1)
|
||||
throw runtime_error("Base64 conversion failed: BIO_write,flush");
|
||||
|
||||
char *data_b64_c;
|
||||
size_t data_b64_length;
|
||||
|
||||
data_b64_length = BIO_get_mem_data(bio_mem, &data_b64_c);
|
||||
string data_b64(data_b64_c, data_b64_length);
|
||||
|
||||
BIO_free_all(bio_b64);
|
||||
|
||||
return data_b64;
|
||||
}
|
||||
|
||||
static void set_time(Json::Value &thing, int time_created)
|
||||
{
|
||||
thing["time_uploaded"] = Json::Int(time(NULL));
|
||||
thing["time_created"] = time_created;
|
||||
}
|
||||
|
||||
static void payload_telemetry_new(Json::Value &doc,
|
||||
const string &data_b64,
|
||||
const string &callsign,
|
||||
Json::Value &receiver_info)
|
||||
{
|
||||
doc["data"] = Json::Value(Json::objectValue);
|
||||
doc["receivers"] = Json::Value(Json::objectValue);
|
||||
doc["type"] = "payload_telemetry";
|
||||
|
||||
doc["data"]["_raw"] = data_b64;
|
||||
doc["receivers"][callsign] = receiver_info;
|
||||
}
|
||||
|
||||
static void payload_telemetry_merge(Json::Value &doc,
|
||||
const string &data_b64,
|
||||
const string &callsign,
|
||||
Json::Value &receiver_info)
|
||||
{
|
||||
string other_b64 = doc["data"]["_raw"].asString();
|
||||
|
||||
if (!other_b64.length() || other_b64 != data_b64)
|
||||
throw CollisionError();
|
||||
|
||||
if (!doc["receivers"].isObject())
|
||||
throw runtime_error("Server gave us an invalid payload telemetry doc");
|
||||
|
||||
doc["receivers"][callsign] = receiver_info;
|
||||
}
|
||||
|
||||
string Uploader::payload_telemetry(const string &data,
|
||||
const Json::Value &metadata,
|
||||
int time_created)
|
||||
{
|
||||
EZ::MutexLock lock(mutex);
|
||||
|
||||
if (!data.length())
|
||||
throw runtime_error("Can't upload string of zero length");
|
||||
|
||||
string data_b64 = base64(data);
|
||||
string doc_id = sha256hex(data_b64);
|
||||
|
||||
if (time_created == -1)
|
||||
time_created = time(NULL);
|
||||
|
||||
Json::Value receiver_info;
|
||||
|
||||
if (metadata.isObject())
|
||||
{
|
||||
if (metadata.isMember("time_created") ||
|
||||
metadata.isMember("time_uploaded") ||
|
||||
metadata.isMember("latest_listener_info") ||
|
||||
metadata.isMember("latest_listener_telemetry"))
|
||||
{
|
||||
throw invalid_argument("found forbidden key in metadata");
|
||||
}
|
||||
|
||||
/* This copies metadata. */
|
||||
receiver_info = metadata;
|
||||
}
|
||||
else if (!metadata.isNull())
|
||||
{
|
||||
throw invalid_argument("metadata must be an object/dict or null");
|
||||
}
|
||||
|
||||
if (latest_listener_info.length())
|
||||
receiver_info["latest_listener_info"] = latest_listener_info;
|
||||
|
||||
if (latest_listener_telemetry.length())
|
||||
receiver_info["latest_listener_telemetry"] = latest_listener_telemetry;
|
||||
|
||||
try
|
||||
{
|
||||
Json::Value doc(Json::objectValue);
|
||||
set_time(receiver_info, time_created);
|
||||
payload_telemetry_new(doc, data_b64, callsign, receiver_info);
|
||||
doc["_id"] = doc_id;
|
||||
database.save_doc(doc);
|
||||
return doc_id;
|
||||
}
|
||||
catch (CouchDB::Conflict e)
|
||||
{
|
||||
for (int attempts = 0; attempts < max_merge_attempts; attempts++)
|
||||
{
|
||||
try
|
||||
{
|
||||
Json::Value *doc = database[doc_id];
|
||||
auto_ptr<Json::Value> doc_destroyer(doc);
|
||||
|
||||
set_time(receiver_info, time_created);
|
||||
payload_telemetry_merge(*doc, data_b64, callsign,
|
||||
receiver_info);
|
||||
database.save_doc(*doc);
|
||||
|
||||
return doc_id;
|
||||
}
|
||||
catch (CouchDB::Conflict e)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
throw UnmergeableError();
|
||||
}
|
||||
}
|
||||
|
||||
string Uploader::listener_doc(const char *type, const Json::Value &data,
|
||||
int time_created)
|
||||
{
|
||||
if (time_created == -1)
|
||||
time_created = time(NULL);
|
||||
|
||||
if (!data.isObject())
|
||||
throw invalid_argument("data must be an object/dict");
|
||||
|
||||
if (data.isMember("callsign"))
|
||||
throw invalid_argument("forbidden key in data");
|
||||
|
||||
Json::Value copied_data(data);
|
||||
copied_data["callsign"] = callsign;
|
||||
|
||||
Json::Value doc(Json::objectValue);
|
||||
doc["data"] = copied_data;
|
||||
doc["type"] = type;
|
||||
|
||||
set_time(doc, time_created);
|
||||
database.save_doc(doc);
|
||||
|
||||
return doc["_id"].asString();
|
||||
}
|
||||
|
||||
string Uploader::listener_telemetry(const Json::Value &data, int time_created)
|
||||
{
|
||||
EZ::MutexLock lock(mutex);
|
||||
|
||||
latest_listener_telemetry =
|
||||
listener_doc("listener_telemetry", data, time_created);
|
||||
return latest_listener_telemetry;
|
||||
}
|
||||
|
||||
string Uploader::listener_info(const Json::Value &data, int time_created)
|
||||
{
|
||||
EZ::MutexLock lock(mutex);
|
||||
|
||||
latest_listener_info =
|
||||
listener_doc("listener_info", data, time_created);
|
||||
return latest_listener_info;
|
||||
}
|
||||
|
||||
vector<Json::Value> *Uploader::flights()
|
||||
{
|
||||
map<string,string> options;
|
||||
ostringstream timefmt;
|
||||
timefmt << time(NULL);
|
||||
|
||||
options["include_docs"] = "true";
|
||||
options["startkey"] = timefmt.str();
|
||||
|
||||
Json::Value *response = database.view("uploader_v1", "flights", options);
|
||||
auto_ptr<Json::Value> response_destroyer(response);
|
||||
|
||||
vector<Json::Value> *result = new vector<Json::Value>;
|
||||
auto_ptr< vector<Json::Value> > result_destroyer(result);
|
||||
|
||||
const Json::Value &rows = (*response)["rows"];
|
||||
Json::Value::const_iterator it;
|
||||
|
||||
result->reserve(rows.size());
|
||||
|
||||
for (it = rows.begin(); it != rows.end(); it++)
|
||||
{
|
||||
result->push_back((*it)["doc"]);
|
||||
}
|
||||
|
||||
result_destroyer.release();
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
} /* namespace habitat */
|
||||
|
|
@ -0,0 +1,61 @@
|
|||
/* Copyright 2011 (C) Daniel Richman. License: GNU GPL 3; see LICENSE. */
|
||||
|
||||
#ifndef HABITAT_UPLOADER_H
|
||||
#define HABITAT_UPLOADER_H
|
||||
|
||||
#include <iostream>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
#include <stdexcept>
|
||||
#include <jsoncpp/json.h>
|
||||
#include "EZ.h"
|
||||
#include "CouchDB.h"
|
||||
|
||||
using namespace std;
|
||||
|
||||
namespace habitat {
|
||||
|
||||
class UnmergeableError : public runtime_error
|
||||
{
|
||||
public:
|
||||
UnmergeableError() : runtime_error("habitat::UnmergeableError") {};
|
||||
UnmergeableError(const string &what) : runtime_error(what) {};
|
||||
};
|
||||
|
||||
class CollisionError : public runtime_error
|
||||
{
|
||||
public:
|
||||
CollisionError() : runtime_error("habitat::CollisionError") {};
|
||||
CollisionError(const string &what) : runtime_error(what) {};
|
||||
};
|
||||
|
||||
class Uploader
|
||||
{
|
||||
EZ::Mutex mutex;
|
||||
const string callsign;
|
||||
CouchDB::Server server;
|
||||
CouchDB::Database database;
|
||||
const int max_merge_attempts;
|
||||
string latest_listener_info;
|
||||
string latest_listener_telemetry;
|
||||
|
||||
string listener_doc(const char *type, const Json::Value &data,
|
||||
int time_created);
|
||||
|
||||
public:
|
||||
Uploader(const string &callsign,
|
||||
const string &couch_uri="http://habitat.habhub.org",
|
||||
const string &couch_db="habitat",
|
||||
int max_merge_attempts=20);
|
||||
~Uploader() {};
|
||||
string payload_telemetry(const string &data,
|
||||
const Json::Value &metadata=Json::Value::null,
|
||||
int time_created=-1);
|
||||
string listener_telemetry(const Json::Value &data, int time_created=-1);
|
||||
string listener_info(const Json::Value &data, int time_created=-1);
|
||||
vector<Json::Value> *flights();
|
||||
};
|
||||
|
||||
} /* namespace habitat */
|
||||
|
||||
#endif /* HABITAT_UPLOADER_H */
|
||||
|
|
@ -0,0 +1,263 @@
|
|||
/* Copyright 2011 (C) Daniel Richman. License: GNU GPL 3; see LICENSE. */
|
||||
|
||||
#include "UploaderThread.h"
|
||||
#include <stdexcept>
|
||||
#include <sstream>
|
||||
|
||||
namespace habitat {
|
||||
|
||||
void UploaderAction::check(habitat::Uploader *u)
|
||||
{
|
||||
if (u == NULL)
|
||||
throw runtime_error("Uploader settings were not initialised");
|
||||
}
|
||||
|
||||
void UploaderSettings::apply(UploaderThread &uthr)
|
||||
{
|
||||
uthr.uploader.reset(new habitat::Uploader(
|
||||
callsign, couch_uri, couch_db, max_merge_attempts));
|
||||
uthr.initialised();
|
||||
}
|
||||
|
||||
string UploaderSettings::describe()
|
||||
{
|
||||
stringstream ss(stringstream::out);
|
||||
ss << "Uploader('" << callsign << "', '" << couch_uri << "', '"
|
||||
<< couch_db << "', " << max_merge_attempts << ")";
|
||||
return ss.str();
|
||||
}
|
||||
|
||||
void UploaderReset::apply(UploaderThread &uthr)
|
||||
{
|
||||
uthr.uploader.reset();
|
||||
uthr.reset_done();
|
||||
}
|
||||
|
||||
string UploaderReset::describe()
|
||||
{
|
||||
return "~Uploader()";
|
||||
}
|
||||
|
||||
void UploaderPayloadTelemetry::apply(UploaderThread &uthr)
|
||||
{
|
||||
check(uthr.uploader.get());
|
||||
|
||||
string result;
|
||||
result = uthr.uploader->payload_telemetry(data, metadata, time_created);
|
||||
|
||||
uthr.saved_id("payload_telemetry", result);
|
||||
}
|
||||
|
||||
string UploaderPayloadTelemetry::describe()
|
||||
{
|
||||
stringstream ss(stringstream::out);
|
||||
Json::FastWriter writer;
|
||||
string metadata_json = writer.write(metadata);
|
||||
metadata_json.erase(metadata_json.length() - 1, 1);
|
||||
ss << "Uploader.payload_telemetry('" << data << "', "
|
||||
<< metadata_json << ", " << time_created << ")";
|
||||
return ss.str();
|
||||
}
|
||||
|
||||
void UploaderListenerTelemetry::apply(UploaderThread &uthr)
|
||||
{
|
||||
check(uthr.uploader.get());
|
||||
string result = uthr.uploader->listener_telemetry(data, time_created);
|
||||
uthr.saved_id("listener_telemetry", result);
|
||||
}
|
||||
|
||||
string UploaderListenerTelemetry::describe()
|
||||
{
|
||||
stringstream ss(stringstream::out);
|
||||
Json::FastWriter writer;
|
||||
string data_json = writer.write(data);
|
||||
data_json.erase(data_json.length() - 1, 1);
|
||||
ss << "Uploader.listener_telemetry(" << data_json << ", "
|
||||
<< time_created << ")";
|
||||
return ss.str();
|
||||
}
|
||||
|
||||
void UploaderListenerInfo::apply(UploaderThread &uthr)
|
||||
{
|
||||
check(uthr.uploader.get());
|
||||
string result = uthr.uploader->listener_info(data, time_created);
|
||||
uthr.saved_id("listener_info", result);
|
||||
}
|
||||
|
||||
string UploaderListenerInfo::describe()
|
||||
{
|
||||
stringstream ss(stringstream::out);
|
||||
Json::FastWriter writer;
|
||||
string data_json = writer.write(data);
|
||||
data_json.erase(data_json.length() - 1, 1);
|
||||
ss << "Uploader.listener_info(" << data_json << ", "
|
||||
<< time_created << ")";
|
||||
return ss.str();
|
||||
}
|
||||
|
||||
void UploaderFlights::apply(UploaderThread &uthr)
|
||||
{
|
||||
check(uthr.uploader.get());
|
||||
auto_ptr< vector<Json::Value> > flights;
|
||||
flights.reset(uthr.uploader->flights());
|
||||
uthr.got_flights(*flights);
|
||||
}
|
||||
|
||||
string UploaderFlights::describe()
|
||||
{
|
||||
return "Uploader.flights()";
|
||||
}
|
||||
|
||||
void UploaderShutdown::apply(UploaderThread &uthr)
|
||||
{
|
||||
throw this;
|
||||
}
|
||||
|
||||
string UploaderShutdown::describe()
|
||||
{
|
||||
return "Shutdown";
|
||||
}
|
||||
|
||||
UploaderThread::UploaderThread() : queued_shutdown(false) {}
|
||||
|
||||
UploaderThread::~UploaderThread()
|
||||
{
|
||||
EZ::MutexLock lock(mutex);
|
||||
|
||||
if (!queued_shutdown)
|
||||
shutdown();
|
||||
|
||||
join();
|
||||
}
|
||||
|
||||
void UploaderThread::queue_action(UploaderAction *action)
|
||||
{
|
||||
auto_ptr<UploaderAction> destroyer(action);
|
||||
|
||||
log("Queuing " + action->describe());
|
||||
queue.put(action);
|
||||
destroyer.release();
|
||||
}
|
||||
|
||||
void UploaderThread::settings(const string &callsign, const string &couch_uri,
|
||||
const string &couch_db, int max_merge_attempts)
|
||||
{
|
||||
queue_action(
|
||||
new UploaderSettings(callsign, couch_uri, couch_db, max_merge_attempts)
|
||||
);
|
||||
}
|
||||
|
||||
void UploaderThread::reset()
|
||||
{
|
||||
queue_action(new UploaderReset());
|
||||
}
|
||||
|
||||
void UploaderThread::payload_telemetry(const string &data,
|
||||
const Json::Value &metadata,
|
||||
int time_created)
|
||||
{
|
||||
queue_action(new UploaderPayloadTelemetry(data, metadata, time_created));
|
||||
}
|
||||
|
||||
void UploaderThread::listener_telemetry(const Json::Value &data,
|
||||
int time_created)
|
||||
{
|
||||
queue_action(new UploaderListenerTelemetry(data, time_created));
|
||||
}
|
||||
|
||||
void UploaderThread::listener_info(const Json::Value &data,
|
||||
int time_created)
|
||||
{
|
||||
queue_action(new UploaderListenerInfo(data, time_created));
|
||||
}
|
||||
|
||||
void UploaderThread::flights()
|
||||
{
|
||||
queue_action(new UploaderFlights());
|
||||
}
|
||||
|
||||
void UploaderThread::shutdown()
|
||||
{
|
||||
/* Borrow the SimpleThread mutex to make queued_shutdown access safe */
|
||||
EZ::MutexLock lock(mutex);
|
||||
|
||||
if (!queued_shutdown)
|
||||
{
|
||||
queue_action(new UploaderShutdown());
|
||||
queued_shutdown = true;
|
||||
}
|
||||
}
|
||||
|
||||
void *UploaderThread::run()
|
||||
{
|
||||
log("Started");
|
||||
|
||||
for (;;)
|
||||
{
|
||||
auto_ptr<UploaderAction> action(queue.get());
|
||||
|
||||
log("Running " + action->describe());
|
||||
|
||||
try
|
||||
{
|
||||
action->apply(*this);
|
||||
}
|
||||
catch (UploaderShutdown *s)
|
||||
{
|
||||
break;
|
||||
}
|
||||
catch (runtime_error e)
|
||||
{
|
||||
caught_exception(e);
|
||||
continue;
|
||||
}
|
||||
catch (invalid_argument e)
|
||||
{
|
||||
caught_exception(e);
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
log("Shutting down");
|
||||
|
||||
return NULL;
|
||||
}
|
||||
|
||||
void UploaderThread::warning(const string &message)
|
||||
{
|
||||
log("Warning: " + message);
|
||||
}
|
||||
|
||||
void UploaderThread::saved_id(const string &type, const string &id)
|
||||
{
|
||||
log("Saved " + type + " doc: " + id);
|
||||
}
|
||||
|
||||
void UploaderThread::initialised()
|
||||
{
|
||||
log("Initialised Uploader");
|
||||
}
|
||||
|
||||
void UploaderThread::reset_done()
|
||||
{
|
||||
log("Settings reset");
|
||||
}
|
||||
|
||||
void UploaderThread::caught_exception(const runtime_error &error)
|
||||
{
|
||||
const string what(error.what());
|
||||
warning("Caught runtime_error: " + what);
|
||||
}
|
||||
|
||||
void UploaderThread::caught_exception(const invalid_argument &error)
|
||||
{
|
||||
const string what(error.what());
|
||||
warning("Caught invalid_argument: " + what);
|
||||
}
|
||||
|
||||
void UploaderThread::got_flights(const vector<Json::Value> &flights)
|
||||
{
|
||||
log("Default action: got_flights; discarding.");
|
||||
}
|
||||
|
||||
} /* namespace habitat */
|
||||
|
|
@ -0,0 +1,186 @@
|
|||
/* Copyright 2011 (C) Daniel Richman. License: GNU GPL 3; see LICENSE. */
|
||||
|
||||
#ifndef HABITAT_UPLOADERTHREAD_H
|
||||
#define HABITAT_UPLOADERTHREAD_H
|
||||
|
||||
#include "EZ.h"
|
||||
#include "Uploader.h"
|
||||
#include <memory>
|
||||
#include <jsoncpp/json.h>
|
||||
|
||||
using namespace std;
|
||||
|
||||
/* Add some more EZ stuff. */
|
||||
namespace habitat {
|
||||
|
||||
class UploaderThread;
|
||||
|
||||
class UploaderAction
|
||||
{
|
||||
protected:
|
||||
void check(habitat::Uploader *u);
|
||||
|
||||
private:
|
||||
virtual void apply(UploaderThread &uthr) = 0;
|
||||
|
||||
friend class UploaderThread;
|
||||
|
||||
public:
|
||||
virtual ~UploaderAction() {};
|
||||
virtual string describe() = 0;
|
||||
};
|
||||
|
||||
class UploaderSettings : public UploaderAction
|
||||
{
|
||||
const string callsign, couch_uri, couch_db;
|
||||
const int max_merge_attempts;
|
||||
|
||||
UploaderSettings(const string &ca, const string &co_u,
|
||||
const string &co_db, int mx)
|
||||
: callsign(ca), couch_uri(co_u), couch_db(co_db),
|
||||
max_merge_attempts(mx)
|
||||
{};
|
||||
~UploaderSettings() {};
|
||||
|
||||
void apply(UploaderThread &uthr);
|
||||
|
||||
friend class UploaderThread;
|
||||
|
||||
public:
|
||||
string describe();
|
||||
};
|
||||
|
||||
class UploaderReset: public UploaderAction
|
||||
{
|
||||
UploaderReset() {};
|
||||
~UploaderReset() {};
|
||||
|
||||
void apply(UploaderThread &uthr);
|
||||
|
||||
friend class UploaderThread;
|
||||
|
||||
public:
|
||||
string describe();
|
||||
};
|
||||
|
||||
class UploaderPayloadTelemetry : public UploaderAction
|
||||
{
|
||||
const string data;
|
||||
const Json::Value metadata;
|
||||
const int time_created;
|
||||
|
||||
UploaderPayloadTelemetry(const string &da, const Json::Value &mda,
|
||||
const int tc)
|
||||
: data(da), metadata(mda), time_created(tc) {};
|
||||
~UploaderPayloadTelemetry() {};
|
||||
|
||||
void apply(UploaderThread &uthr);
|
||||
|
||||
friend class UploaderThread;
|
||||
|
||||
public:
|
||||
string describe();
|
||||
};
|
||||
|
||||
class UploaderListenerTelemetry : public UploaderAction
|
||||
{
|
||||
const Json::Value data;
|
||||
const int time_created;
|
||||
|
||||
UploaderListenerTelemetry(const Json::Value &da, int tc)
|
||||
: data(da), time_created(tc) {};
|
||||
~UploaderListenerTelemetry() {};
|
||||
|
||||
void apply(UploaderThread &uthr);
|
||||
|
||||
friend class UploaderThread;
|
||||
|
||||
public:
|
||||
string describe();
|
||||
};
|
||||
|
||||
class UploaderListenerInfo : public UploaderAction
|
||||
{
|
||||
const Json::Value data;
|
||||
const int time_created;
|
||||
|
||||
UploaderListenerInfo(const Json::Value &da, int tc)
|
||||
: data(da), time_created(tc) {};
|
||||
~UploaderListenerInfo() {};
|
||||
|
||||
void apply(UploaderThread &uthr);
|
||||
|
||||
friend class UploaderThread;
|
||||
|
||||
public:
|
||||
string describe();
|
||||
};
|
||||
|
||||
class UploaderFlights : public UploaderAction
|
||||
{
|
||||
void apply(UploaderThread &uthr);
|
||||
friend class UploaderThread;
|
||||
|
||||
public:
|
||||
string describe();
|
||||
};
|
||||
|
||||
class UploaderShutdown : public UploaderAction
|
||||
{
|
||||
void apply(UploaderThread &uthr);
|
||||
friend class UploaderThread;
|
||||
|
||||
public:
|
||||
string describe();
|
||||
};
|
||||
|
||||
class UploaderThread : public EZ::SimpleThread
|
||||
{
|
||||
EZ::Queue<UploaderAction *> queue;
|
||||
auto_ptr<habitat::Uploader> uploader;
|
||||
|
||||
bool queued_shutdown;
|
||||
|
||||
void queue_action(UploaderAction *ac);
|
||||
|
||||
friend class UploaderAction;
|
||||
friend class UploaderSettings;
|
||||
friend class UploaderReset;
|
||||
friend class UploaderPayloadTelemetry;
|
||||
friend class UploaderListenerTelemetry;
|
||||
friend class UploaderListenerInfo;
|
||||
friend class UploaderFlights;
|
||||
|
||||
public:
|
||||
UploaderThread();
|
||||
virtual ~UploaderThread();
|
||||
|
||||
void settings(const string &callsign,
|
||||
const string &couch_uri="http://habitat.habhub.org",
|
||||
const string &couch_db="habitat",
|
||||
int max_merge_attempts=20);
|
||||
void reset();
|
||||
void payload_telemetry(const string &data,
|
||||
const Json::Value &metadata=Json::Value::null,
|
||||
int time_created=-1);
|
||||
void listener_telemetry(const Json::Value &data, int time_created=-1);
|
||||
void listener_info(const Json::Value &data, int time_created=-1);
|
||||
void flights();
|
||||
void shutdown();
|
||||
|
||||
void *run();
|
||||
void detach();
|
||||
|
||||
virtual void log(const string &message) = 0;
|
||||
virtual void warning(const string &message);
|
||||
virtual void saved_id(const string &type, const string &id);
|
||||
virtual void initialised();
|
||||
virtual void reset_done();
|
||||
virtual void caught_exception(const runtime_error &error);
|
||||
virtual void caught_exception(const invalid_argument &error);
|
||||
virtual void got_flights(const vector<Json::Value> &flights);
|
||||
};
|
||||
|
||||
} /* namespace habitat */
|
||||
|
||||
#endif /* HABITAT_UPLOADERTHREAD_H */
|
||||
|
|
@ -0,0 +1,335 @@
|
|||
import subprocess
|
||||
import json
|
||||
import fcntl
|
||||
import os
|
||||
import errno
|
||||
|
||||
# Mox-esque class that is 'equal' to another string if the value it is
|
||||
# initialised is contained in that string; used to avoid writing out the
|
||||
# whole of check_status()
|
||||
class EqualIfIn:
|
||||
def __init__(self, test):
|
||||
self.test = test
|
||||
def __eq__(self, rhs):
|
||||
return isinstance(rhs, basestring) and self.test.lower() in rhs.lower()
|
||||
def __repr__(self):
|
||||
return "<EqIn " + repr(self.test) + ">"
|
||||
|
||||
class Proxy:
|
||||
def __init__(self, command):
|
||||
self.closed = False
|
||||
self.p = subprocess.Popen(command, stdin=subprocess.PIPE,
|
||||
stdout=subprocess.PIPE)
|
||||
|
||||
def _write(self, command):
|
||||
print ">>", repr(command)
|
||||
self.p.stdin.write(json.dumps(command))
|
||||
self.p.stdin.write("\n")
|
||||
|
||||
def _read(self):
|
||||
line = self.p.stdout.readline()
|
||||
assert line and line.endswith("\n")
|
||||
obj = json.loads(line)
|
||||
|
||||
print "<<", repr(obj)
|
||||
return obj
|
||||
|
||||
def check_quiet(self):
|
||||
fd = self.p.stdout.fileno()
|
||||
fl = fcntl.fcntl(fd, fcntl.F_GETFL)
|
||||
fcntl.fcntl(fd, fcntl.F_SETFL, fl | os.O_NONBLOCK)
|
||||
|
||||
try:
|
||||
line = self.p.stdout.readline()
|
||||
print ">>", line
|
||||
except IOError as e:
|
||||
if e.errno != errno.EAGAIN:
|
||||
raise
|
||||
else:
|
||||
# line is '' when EOF; '\n' is an empty line
|
||||
if line != '':
|
||||
raise AssertionError("expected IOError(EAGAIN), not " +
|
||||
repr(line))
|
||||
|
||||
fcntl.fcntl(fd, fcntl.F_SETFL, fl)
|
||||
|
||||
def add(self, name):
|
||||
self._write(["add", name])
|
||||
|
||||
def skipped(self, num):
|
||||
self._write(["skipped", num])
|
||||
|
||||
def push(self, data):
|
||||
for char in data:
|
||||
self._write(["push", char])
|
||||
|
||||
def set_current_payload(self, value):
|
||||
self._write(["set_current_payload", value])
|
||||
|
||||
def check(self, match):
|
||||
obj = self._read()
|
||||
assert len(obj) >= len(match)
|
||||
assert obj[:len(match)] == match
|
||||
|
||||
def _check_type(self, name, arg):
|
||||
if arg:
|
||||
self.check([name, arg])
|
||||
else:
|
||||
self.check([name])
|
||||
|
||||
def check_status(self, message=None):
|
||||
if message:
|
||||
message = EqualIfIn(message)
|
||||
self._check_type("status", message)
|
||||
|
||||
def check_data(self, data=None):
|
||||
self._check_type("data", data)
|
||||
|
||||
def check_upload(self, data=None):
|
||||
self._check_type("upload", data)
|
||||
|
||||
def __del__(self):
|
||||
if not self.closed:
|
||||
self.close(check=False)
|
||||
|
||||
def close(self, check=True):
|
||||
self.closed = True
|
||||
self.p.stdin.close()
|
||||
ret = self.p.wait()
|
||||
|
||||
if check:
|
||||
self.check_quiet()
|
||||
assert ret == 0
|
||||
|
||||
class TestExtractorManager:
|
||||
def setup(self):
|
||||
self.extr = Proxy("tests/extractor")
|
||||
|
||||
def teardown(self):
|
||||
self.extr.close()
|
||||
|
||||
def test_management(self):
|
||||
self.extr.push("$$this,is,a,string\n")
|
||||
self.extr.check_quiet()
|
||||
|
||||
self.extr.add("UKHASExtractor")
|
||||
self.extr.push("$$this,is,a,string\n")
|
||||
self.extr.check_status("start delim")
|
||||
self.extr.check_upload()
|
||||
self.extr.check_status("extracted")
|
||||
self.extr.check_status("parse failed")
|
||||
self.extr.check_data()
|
||||
|
||||
class TestUKHASExtractor:
|
||||
def setup(self):
|
||||
self.extr = Proxy("tests/extractor")
|
||||
self.extr.add("UKHASExtractor")
|
||||
|
||||
def teardown(self):
|
||||
self.extr.close()
|
||||
|
||||
def test_finds_start_delimiter(self):
|
||||
self.extr.push("$")
|
||||
self.extr.check_quiet()
|
||||
self.extr.push("$")
|
||||
self.extr.check_status("start delim")
|
||||
|
||||
def test_extracts(self):
|
||||
string = "$$a,simple,test*00\n"
|
||||
self.extr.check_quiet()
|
||||
self.extr.push(string)
|
||||
self.extr.check_status("start delim")
|
||||
self.extr.check_upload(string)
|
||||
self.extr.check_status("extracted")
|
||||
self.extr.check_status("parse failed")
|
||||
self.extr.check_data({"_sentence": string})
|
||||
|
||||
def test_can_restart(self):
|
||||
self.extr.push("this is some garbage just to mess things up")
|
||||
self.extr.check_quiet()
|
||||
self.extr.push("$$")
|
||||
self.extr.check_status("start delim")
|
||||
|
||||
self.extr.push("garbage: after seeing the delimiter, we lose signal.")
|
||||
self.extr.push("some extra $s to con$fuse it $")
|
||||
self.extr.push("$$")
|
||||
self.extr.check_status("start delim")
|
||||
self.extr.check_status("start delim")
|
||||
self.extr.check_quiet()
|
||||
self.extr.push("helloworld")
|
||||
self.extr.check_quiet()
|
||||
self.extr.push("\n")
|
||||
self.extr.check_upload("$$helloworld\n")
|
||||
self.extr.check_status("extracted")
|
||||
self.extr.check_status("parse failed")
|
||||
self.extr.check_data()
|
||||
|
||||
def test_gives_up_after_1k(self):
|
||||
self.extr.push("$$")
|
||||
self.extr.check_status("start delim")
|
||||
|
||||
self.extr.push("a" * 1022)
|
||||
self.extr.check_status("giving up")
|
||||
self.extr.check_quiet()
|
||||
|
||||
# Should have given up, so a \n won't cause an upload:
|
||||
self.extr.push("\n")
|
||||
self.extr.check_quiet()
|
||||
|
||||
self.test_extracts()
|
||||
|
||||
def test_gives_up_after_16skipped(self):
|
||||
self.extr.push("$$")
|
||||
self.extr.check_status("start delim")
|
||||
self.extr.skipped(10000)
|
||||
self.extr.check_status("giving up")
|
||||
self.extr.check_quiet()
|
||||
self.extr.push("\n")
|
||||
self.extr.check_quiet()
|
||||
|
||||
def test_gives_up_after_16garbage(self):
|
||||
self.extr.push("$$")
|
||||
self.extr.check_status("start delim")
|
||||
|
||||
self.extr.push("some,legit,data")
|
||||
self.extr.push("\t some printable data" * 17)
|
||||
self.extr.check_status("giving up")
|
||||
self.extr.check_quiet()
|
||||
|
||||
self.extr.push("\n")
|
||||
self.extr.check_quiet()
|
||||
|
||||
self.test_extracts()
|
||||
|
||||
def test_skipped(self):
|
||||
self.extr.check_quiet()
|
||||
self.extr.push("$$some")
|
||||
self.extr.check_status("start delim")
|
||||
self.extr.skipped(5)
|
||||
self.extr.push("data\n")
|
||||
# JsonCPP doesn't support \0 in strings, so the mock UploaderThread
|
||||
# replaces it with \1s
|
||||
self.extr.check_upload("$$some\1\1\1\1\1data\n")
|
||||
self.extr.check_status("extracted")
|
||||
self.extr.check_status("parse failed")
|
||||
self.extr.check_data()
|
||||
|
||||
def basic_data_dict(self, string, callsign):
|
||||
return {"_sentence": string, "_parsed": True, "_basic": True,
|
||||
"_protocol": "UKHAS", "payload": callsign}
|
||||
|
||||
def check_noconfig(self, string, callsign):
|
||||
self.extr.push(string)
|
||||
self.extr.check_status("start delim")
|
||||
self.extr.check_upload(string)
|
||||
self.extr.check_status("extracted")
|
||||
self.extr.check_data(self.basic_data_dict(string, callsign))
|
||||
|
||||
def test_crude_parse_noconfig_xor(self):
|
||||
self.check_noconfig("$$mypayload,has,a,valid,checksum*1a\n",
|
||||
"mypayload")
|
||||
|
||||
def test_crude_parse_noconfig_crc16_ccitt(self):
|
||||
self.check_noconfig("$$mypayload,has,a,valid,checksum*1018\n",
|
||||
"mypayload")
|
||||
|
||||
crude_parse_flight_doc = {
|
||||
"sentence": {
|
||||
"payload": "TESTING",
|
||||
"checksum": "crc16-ccitt",
|
||||
"fields": [
|
||||
{"name": "field_a"},
|
||||
{"name": "field_b"},
|
||||
{"name": "field_c"}
|
||||
],
|
||||
}
|
||||
}
|
||||
|
||||
def test_crude_parse_config(self):
|
||||
self.extr.set_current_payload(self.crude_parse_flight_doc)
|
||||
string = "$$TESTING,value_a,value_b,value_c*8C3E\n"
|
||||
self.extr.push(string)
|
||||
self.extr.check_status("start delim")
|
||||
self.extr.check_upload(string)
|
||||
self.extr.check_status("extracted")
|
||||
self.extr.check_data({"_sentence": string, "_parsed": True,
|
||||
"_protocol": "UKHAS", "payload": "TESTING",
|
||||
"field_a": "value_a", "field_b": "value_b",
|
||||
"field_c": "value_c"})
|
||||
|
||||
def test_crude_checks(self):
|
||||
checks = [
|
||||
("$$TESTING,a,b,c*asdfg\n", "invalid checksum len"),
|
||||
("$$TESTING,a,b,c*45\n", "invalid checksum: expected 1A"),
|
||||
("$$TESTING,a,b,c*AAAA\n", "invalid checksum: expected BEBC"),
|
||||
("$$TESTING,val_a,val_b*4EB7\n", "incorrect number of fields"),
|
||||
("$$TESTING,a,b,c*1A\n", "wrong checksum type"),
|
||||
("$$ANOTHER,a,b,c*2355\n", "incorrect callsign"),
|
||||
]
|
||||
|
||||
self.extr.set_current_payload(self.crude_parse_flight_doc)
|
||||
|
||||
for (string, error) in checks:
|
||||
self.extr.push(string)
|
||||
self.extr.check_status("start delim")
|
||||
self.extr.check_upload(string)
|
||||
self.extr.check_status("extracted")
|
||||
self.extr.check_status(error)
|
||||
self.extr.check_data()
|
||||
|
||||
multi_config_flight_doc = {
|
||||
"sentence": [
|
||||
{ "payload": "AWKWARD",
|
||||
"checksum": "crc16-ccitt",
|
||||
"fields": [ {"name": "fa"}, {"name": "fo"}, {"name": "fc"} ] },
|
||||
{ "payload": "AWKWARD",
|
||||
"checksum": "crc16-ccitt",
|
||||
"fields": [ {"name": "fa"}, {"name": "fb"} ] }
|
||||
]
|
||||
}
|
||||
|
||||
def test_multi_config(self):
|
||||
self.extr.set_current_payload(self.multi_config_flight_doc)
|
||||
string = "$$AWKWARD,hello,world*D4E9\n"
|
||||
self.extr.push(string)
|
||||
self.extr.check_status("start delim")
|
||||
self.extr.check_upload(string)
|
||||
self.extr.check_status("extracted")
|
||||
self.extr.check_data({"_sentence": string, "_parsed": True,
|
||||
"_protocol": "UKHAS", "payload": "AWKWARD",
|
||||
"fa": "hello", "fb": "world"})
|
||||
|
||||
string = "$$AWKWARD,extended,other,data*F01F\n"
|
||||
self.extr.push(string)
|
||||
self.extr.check_status("start delim")
|
||||
self.extr.check_upload(string)
|
||||
self.extr.check_status("extracted")
|
||||
self.extr.check_data({"_sentence": string, "_parsed": True,
|
||||
"_protocol": "UKHAS", "payload": "AWKWARD",
|
||||
"fa": "extended", "fo": "other", "fc": "data"})
|
||||
|
||||
ddmmmmmm_flight_doc = {
|
||||
"sentence": {
|
||||
"payload": "TESTING",
|
||||
"checksum": "crc16-ccitt",
|
||||
"fields": [
|
||||
{"sensor":"stdtelem.coordinate","name":"lat_a",
|
||||
"format":"dd.dddd"},
|
||||
{"sensor":"stdtelem.coordinate","name":"lat_b",
|
||||
"format":"ddmm.mm"},
|
||||
{"name": "field_b"}
|
||||
],
|
||||
}
|
||||
}
|
||||
|
||||
def test_ddmmmmmm(self):
|
||||
self.extr.set_current_payload(self.ddmmmmmm_flight_doc)
|
||||
string = "$$TESTING,0024.124583,5116.5271,whatever*14BA\n"
|
||||
self.extr.push(string)
|
||||
self.extr.check_status("start delim")
|
||||
self.extr.check_upload(string)
|
||||
self.extr.check_status("extracted")
|
||||
self.extr.check_data({"_sentence": string, "_parsed": True,
|
||||
"_protocol": "UKHAS", "payload": "TESTING",
|
||||
"lat_a": "0024.124583", "lat_b": "51.27545",
|
||||
"field_b": "whatever" })
|
||||
|
|
@ -0,0 +1,111 @@
|
|||
/* Copyright 2011 (C) Daniel Richman. License: GNU GPL 3; see LICENSE. */
|
||||
|
||||
#include <iostream>
|
||||
#include <stdexcept>
|
||||
#include <jsoncpp/json.h>
|
||||
#include <memory>
|
||||
|
||||
#include "Extractor.h"
|
||||
#include "UKHASExtractor.h"
|
||||
|
||||
using namespace std;
|
||||
|
||||
class JsonIOExtractorManager : public habitat::ExtractorManager
|
||||
{
|
||||
void write(const string &name, const Json::Value &arg)
|
||||
{
|
||||
Json::Value root(Json::arrayValue);
|
||||
root.append(name);
|
||||
root.append(arg);
|
||||
|
||||
Json::FastWriter writer;
|
||||
cout << writer.write(root);
|
||||
};
|
||||
|
||||
public:
|
||||
JsonIOExtractorManager(habitat::UploaderThread &u)
|
||||
: habitat::ExtractorManager(u) {};
|
||||
void status(const string &msg) { write("status", msg); };
|
||||
void data(const Json::Value &d) { write("data", d); };
|
||||
};
|
||||
|
||||
void handle_command(const Json::Value &command,
|
||||
JsonIOExtractorManager &manager,
|
||||
habitat::UKHASExtractor &extractor,
|
||||
auto_ptr<Json::Value> ¤t_payload);
|
||||
|
||||
int main(int argc, char **argv)
|
||||
{
|
||||
habitat::UploaderThread thread;
|
||||
JsonIOExtractorManager manager(thread);
|
||||
habitat::UKHASExtractor extractor;
|
||||
auto_ptr<Json::Value> current_payload;
|
||||
|
||||
for (;;)
|
||||
{
|
||||
char line[1024];
|
||||
cin.getline(line, 1024);
|
||||
|
||||
if (line[0] == '\0')
|
||||
break;
|
||||
|
||||
Json::Reader reader;
|
||||
Json::Value command;
|
||||
|
||||
if (!reader.parse(line, command, false))
|
||||
throw runtime_error("JSON parsing failed");
|
||||
|
||||
if (!command.isArray() || !command[0u].isString())
|
||||
throw runtime_error("Invalid JSON input");
|
||||
|
||||
handle_command(command, manager, extractor, current_payload);
|
||||
}
|
||||
}
|
||||
|
||||
void handle_command(const Json::Value &command,
|
||||
JsonIOExtractorManager &manager,
|
||||
habitat::UKHASExtractor &extractor,
|
||||
auto_ptr<Json::Value> ¤t_payload)
|
||||
{
|
||||
string command_name = command[0u].asString();
|
||||
const Json::Value &arg = command[1u];
|
||||
|
||||
if (command_name == "add")
|
||||
{
|
||||
if (arg != "UKHASExtractor")
|
||||
throw runtime_error("Only UKHASExtractor implemented");
|
||||
|
||||
manager.add(extractor);
|
||||
}
|
||||
else if (command_name == "skipped")
|
||||
{
|
||||
if (!arg.isInt())
|
||||
throw runtime_error("Invalid JSON input");
|
||||
|
||||
manager.skipped(arg.asInt());
|
||||
}
|
||||
else if (command_name == "push")
|
||||
{
|
||||
const Json::Value &arg2 = command[2u];
|
||||
|
||||
if (!arg.isString() || arg.asString().length() != 1)
|
||||
throw runtime_error("Invalid JSON input");
|
||||
|
||||
if (arg2.isInt())
|
||||
manager.push(arg.asString()[0],
|
||||
static_cast<enum habitat::push_flags>(arg2.asInt()));
|
||||
else if (arg2.isNull())
|
||||
manager.push(arg.asString()[0]);
|
||||
else
|
||||
throw runtime_error("Invalid JSON input");
|
||||
}
|
||||
else if (command_name == "set_current_payload")
|
||||
{
|
||||
current_payload.reset(new Json::Value(arg));
|
||||
manager.payload(current_payload.get());
|
||||
}
|
||||
else
|
||||
{
|
||||
throw runtime_error("Invalid JSON input");
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,75 @@
|
|||
#ifndef HABITAT_TEST_EXTRACTOR_MOCKS_H
|
||||
#define HABITAT_TEST_EXTRACTOR_MOCKS_H
|
||||
|
||||
/* Prevent EZ.h inclusion */
|
||||
#define HABITAT_EZ_H
|
||||
|
||||
#include <assert.h>
|
||||
|
||||
namespace EZ {
|
||||
|
||||
/* The tests are single threaded, so this works */
|
||||
|
||||
class Mutex
|
||||
{
|
||||
protected:
|
||||
int lock_count;
|
||||
friend class MutexLock;
|
||||
|
||||
public:
|
||||
Mutex() : lock_count(0) {};
|
||||
~Mutex() {};
|
||||
};
|
||||
|
||||
class MutexLock
|
||||
{
|
||||
Mutex &m;
|
||||
|
||||
public:
|
||||
MutexLock(Mutex &_m) : m(_m) { m.lock_count++; };
|
||||
~MutexLock() { m.lock_count--; assert(m.lock_count >= 0); };
|
||||
};
|
||||
|
||||
}
|
||||
|
||||
/* Prevent inclusion of real UploaderThread */
|
||||
#define HABITAT_UPLOADERTHREAD_H
|
||||
|
||||
#include <iostream>
|
||||
#include <string>
|
||||
#include <algorithm>
|
||||
#include <jsoncpp/json.h>
|
||||
|
||||
namespace habitat {
|
||||
|
||||
static void inplace_nonulls(char &c)
|
||||
{
|
||||
/* JsonCPP doesn't support '\0' in strings :'-( */
|
||||
if (c == '\0')
|
||||
c = '\1';
|
||||
}
|
||||
|
||||
class UploaderThread
|
||||
{
|
||||
public:
|
||||
void payload_telemetry(const std::string &data,
|
||||
const Json::Value &metadata=Json::Value::null,
|
||||
int time_created=-1)
|
||||
{
|
||||
std::string data_copy(data);
|
||||
std::for_each(data_copy.begin(), data_copy.end(), inplace_nonulls);
|
||||
|
||||
Json::Value root(Json::arrayValue);
|
||||
root.append("upload");
|
||||
root.append(data_copy);
|
||||
root.append(metadata);
|
||||
root.append(time_created);
|
||||
|
||||
Json::FastWriter writer;
|
||||
std::cout << writer.write(root);
|
||||
};
|
||||
};
|
||||
|
||||
} /* namespace habitat */
|
||||
|
||||
#endif /* HABITAT_TEST_EXTRACTOR_MOCK_UPLOADERTHREAD_H */
|
||||
|
|
@ -0,0 +1,845 @@
|
|||
# Copyright 2011 (C) Daniel Richman. License: GNU GPL 3; see LICENSE
|
||||
|
||||
import subprocess
|
||||
import os
|
||||
import errno
|
||||
import fcntl
|
||||
import tempfile
|
||||
import json
|
||||
import BaseHTTPServer
|
||||
import threading
|
||||
import collections
|
||||
import time
|
||||
import uuid
|
||||
import copy
|
||||
|
||||
try:
|
||||
import elementtree.ElementTree
|
||||
except:
|
||||
elementtree = None
|
||||
|
||||
class ProxyException:
|
||||
def __init__(self, name, what=None):
|
||||
self.name = name
|
||||
self.what = what
|
||||
|
||||
def __str__(self):
|
||||
return "ProxyException: {0.name}: {0.what!r}".format(self)
|
||||
|
||||
class Callbacks:
|
||||
def __init__(self):
|
||||
self.lock = threading.RLock()
|
||||
self.fake_time = 10000 # set in fake_main.cxx
|
||||
|
||||
def advance_time(self, amount=1):
|
||||
with self.lock:
|
||||
self.fake_time += amount
|
||||
|
||||
def time(self):
|
||||
with self.lock:
|
||||
return self.fake_time
|
||||
|
||||
def time_project(self, value):
|
||||
"""what the time will be value seconds into the future"""
|
||||
return 10000 + value
|
||||
|
||||
class Proxy:
|
||||
def __init__(self, command, callsign, couch_uri=None, couch_db=None,
|
||||
max_merge_attempts=None, callbacks=None, with_valgrind=False):
|
||||
|
||||
self.closed = False
|
||||
self.blocking = True
|
||||
|
||||
if with_valgrind:
|
||||
self.xmlfile = tempfile.NamedTemporaryFile("a+b")
|
||||
args = ("valgrind", "--quiet", "--xml=yes",
|
||||
"--xml-file=" + self.xmlfile.name, command)
|
||||
self.p = subprocess.Popen(args, stdin=subprocess.PIPE,
|
||||
stdout=subprocess.PIPE)
|
||||
else:
|
||||
self.xmlfile = None
|
||||
self.p = subprocess.Popen(command, stdin=subprocess.PIPE,
|
||||
stdout=subprocess.PIPE)
|
||||
|
||||
|
||||
self.callbacks = callbacks
|
||||
self.re_init(callsign, couch_uri, couch_db, max_merge_attempts=None)
|
||||
|
||||
def re_init(self, callsign, couch_uri=None, couch_db=None,
|
||||
max_merge_attempts=None):
|
||||
init_args = ["init", callsign]
|
||||
|
||||
for a in [couch_uri, couch_db, max_merge_attempts]:
|
||||
if a is None:
|
||||
break
|
||||
init_args.append(a)
|
||||
|
||||
self._proxy(init_args)
|
||||
|
||||
def _write(self, command):
|
||||
print ">>", repr(command)
|
||||
self.p.stdin.write(json.dumps(command))
|
||||
self.p.stdin.write("\n")
|
||||
|
||||
def _read(self):
|
||||
line = self.p.stdout.readline()
|
||||
assert line and line.endswith("\n")
|
||||
obj = json.loads(line)
|
||||
|
||||
print "<<", repr(obj)
|
||||
return obj
|
||||
|
||||
def _proxy(self, command):
|
||||
self._write(command)
|
||||
return self.complete()
|
||||
|
||||
def complete(self):
|
||||
while True:
|
||||
obj = self._read()
|
||||
|
||||
if obj[0] == "return":
|
||||
if len(obj) == 1:
|
||||
return None
|
||||
else:
|
||||
return obj[1]
|
||||
elif obj[0] == "error":
|
||||
if len(obj) == 3:
|
||||
raise ProxyException(obj[1], obj[2])
|
||||
else:
|
||||
raise ProxyException(obj[1])
|
||||
elif obj[0] == "callback":
|
||||
if len(obj) == 3:
|
||||
(cb, name, args) = obj
|
||||
else:
|
||||
(cb, name) = obj
|
||||
args = []
|
||||
func = getattr(self.callbacks, name)
|
||||
result = func(*args)
|
||||
|
||||
self._write(["return", result])
|
||||
elif obj[0] == "log":
|
||||
pass
|
||||
else:
|
||||
raise AssertionError("invalid response")
|
||||
|
||||
def unblock(self):
|
||||
assert self.blocking
|
||||
self.blocking = False
|
||||
|
||||
fd = self.p.stdout.fileno()
|
||||
self.fl = fcntl.fcntl(fd, fcntl.F_GETFL)
|
||||
fcntl.fcntl(fd, fcntl.F_SETFL, self.fl | os.O_NONBLOCK)
|
||||
|
||||
def block(self):
|
||||
assert not self.blocking
|
||||
self.blocking = True
|
||||
|
||||
fd = self.p.stdout.fileno()
|
||||
fcntl.fcntl(fd, fcntl.F_SETFL, self.fl)
|
||||
|
||||
def __del__(self):
|
||||
if not self.closed:
|
||||
self.close(check=False)
|
||||
|
||||
def close(self, check=True):
|
||||
self.closed = True
|
||||
self.p.stdin.close()
|
||||
ret = self.p.wait()
|
||||
|
||||
if check:
|
||||
assert ret == 0, ret
|
||||
self._check_valgrind()
|
||||
|
||||
def _check_valgrind(self):
|
||||
if elementtree == None:
|
||||
raise AssertionError("Need elementtree in order to check Valgrind")
|
||||
|
||||
if self.xmlfile:
|
||||
self.xmlfile.seek(0)
|
||||
tree = elementtree.ElementTree.parse(self.xmlfile)
|
||||
assert tree.find("error") == None
|
||||
|
||||
def payload_telemetry(self, data, *args):
|
||||
return self._proxy(["payload_telemetry", data] + list(args))
|
||||
|
||||
def listener_telemetry(self, data, *args):
|
||||
return self._proxy(["listener_telemetry", data] + list(args))
|
||||
|
||||
def listener_info(self, data, *args):
|
||||
return self._proxy(["listener_info", data] + list(args))
|
||||
|
||||
def flights(self):
|
||||
return self._proxy(["flights"])
|
||||
|
||||
def reset(self):
|
||||
return self._proxy(["reset"])
|
||||
|
||||
temp_port = 51205
|
||||
|
||||
def next_temp_port():
|
||||
global temp_port
|
||||
temp_port += 1
|
||||
return temp_port
|
||||
|
||||
class MockHTTP(BaseHTTPServer.HTTPServer):
|
||||
def __init__(self, server_address=None, callbacks=None):
|
||||
if server_address == None:
|
||||
server_address = ('localhost', next_temp_port())
|
||||
|
||||
BaseHTTPServer.HTTPServer.__init__(self, server_address,
|
||||
MockHTTPHandler)
|
||||
self.expecting = False
|
||||
self.expect_queue = collections.deque()
|
||||
self.url = "http://localhost:{0}".format(self.server_port)
|
||||
self.timeout = 1
|
||||
self.callbacks = callbacks
|
||||
|
||||
def advance_time(self, value):
|
||||
self.callbacks.advance_time(value)
|
||||
|
||||
expect_defaults = {
|
||||
# expect:
|
||||
"method": "GET",
|
||||
"path": "/",
|
||||
"body": None, # string if you expect something from a POST
|
||||
# body_json=object
|
||||
|
||||
# and respond with:
|
||||
"code": 404,
|
||||
"respond": "If this was a 200, this would be your page"
|
||||
# respond_json=object
|
||||
}
|
||||
|
||||
def expect_request(self, **kwargs):
|
||||
assert not self.expecting
|
||||
|
||||
e = self.expect_defaults.copy()
|
||||
e.update(kwargs)
|
||||
|
||||
self.expect_queue.append(e)
|
||||
|
||||
def run(self):
|
||||
assert not self.expecting
|
||||
self.expecting = True
|
||||
self.expect_handled = False
|
||||
self.expect_successes = 0
|
||||
self.expect_length = len(self.expect_queue)
|
||||
|
||||
self.expect_thread = threading.Thread(target=self._run_expect)
|
||||
self.expect_thread.daemon = True
|
||||
self.expect_thread.start()
|
||||
|
||||
def check(self):
|
||||
assert self.expecting
|
||||
self.expect_queue.clear()
|
||||
self.expect_thread.join()
|
||||
|
||||
assert self.expect_successes == self.expect_length
|
||||
|
||||
self.expecting = False
|
||||
|
||||
def _run_expect(self):
|
||||
self.error = None
|
||||
while len(self.expect_queue):
|
||||
self.handle_request()
|
||||
|
||||
class MockHTTPHandler(BaseHTTPServer.BaseHTTPRequestHandler):
|
||||
def compare(self, a, b, what):
|
||||
if a != b:
|
||||
raise AssertionError("http request expect", what, a, b)
|
||||
|
||||
def check_expect(self):
|
||||
print "-- HTTP " + self.command + " " + self.path
|
||||
|
||||
assert self.server.expecting
|
||||
e = self.server.expect_queue.popleft()
|
||||
|
||||
self.compare(e["method"], self.command, "method")
|
||||
self.compare(e["path"], self.path, "path")
|
||||
|
||||
expect_100_header = self.headers.getheader('expect')
|
||||
expect_100 = expect_100_header and \
|
||||
expect_100_header.lower() == "100-continue"
|
||||
support_100 = self.request_version != 'HTTP/0.9'
|
||||
|
||||
if support_100 and expect_100:
|
||||
self.wfile.write(self.protocol_version + " 100 Continue\r\n\r\n")
|
||||
|
||||
length = self.headers.getheader('content-length')
|
||||
if length:
|
||||
length = int(length)
|
||||
body = self.rfile.read(length)
|
||||
assert len(body) == length
|
||||
else:
|
||||
body = None
|
||||
|
||||
if "body_json" in e:
|
||||
self.compare(e["body_json"], json.loads(body), "body_json")
|
||||
else:
|
||||
self.compare(e["body"], body, "body")
|
||||
|
||||
code = e["code"]
|
||||
if "respond_json" in e:
|
||||
content = json.dumps(e["respond_json"])
|
||||
else:
|
||||
content = e["respond"]
|
||||
|
||||
if "wait" in e:
|
||||
e["wait"].set()
|
||||
|
||||
if "delay" in e:
|
||||
e["delay"].wait()
|
||||
|
||||
if "advance_time_after" in e:
|
||||
self.server.advance_time(e["advance_time_after"])
|
||||
|
||||
self.send_response(code)
|
||||
self.send_header("Content-Length", str(len(content)))
|
||||
self.end_headers()
|
||||
self.wfile.write(content)
|
||||
|
||||
self.server.expect_successes += 1
|
||||
|
||||
def log_request(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
do_POST = check_expect
|
||||
do_GET = check_expect
|
||||
do_PUT = check_expect
|
||||
|
||||
class TestCPPConnector:
|
||||
command = "tests/cpp_connector"
|
||||
|
||||
def setup(self):
|
||||
self.callbacks = Callbacks()
|
||||
self.couchdb = MockHTTP(callbacks=self.callbacks)
|
||||
self.uploader = Proxy(self.command, "PROXYCALL", self.couchdb.url,
|
||||
callbacks=self.callbacks, with_valgrind=False)
|
||||
self.uuids = collections.deque()
|
||||
|
||||
self.db_path = "/habitat/"
|
||||
|
||||
def teardown(self):
|
||||
self.uploader.close()
|
||||
self.couchdb.server_close()
|
||||
|
||||
def gen_fake_uuid(self):
|
||||
return str(uuid.uuid1()).replace("-", "")
|
||||
|
||||
def gen_fake_rev(self, num=1):
|
||||
return str(num) + "-" + self.gen_fake_uuid()
|
||||
|
||||
def expect_uuid_request(self):
|
||||
new_uuids = [self.gen_fake_uuid() for i in xrange(100)]
|
||||
self.uuids.extend(new_uuids)
|
||||
|
||||
self.couchdb.expect_request(
|
||||
path="/_uuids?count=100",
|
||||
code=200,
|
||||
respond_json={"uuids": new_uuids},
|
||||
)
|
||||
|
||||
def pop_uuid(self):
|
||||
self.ensure_uuids()
|
||||
return self.uuids.popleft()
|
||||
|
||||
def ensure_uuids(self):
|
||||
if not len(self.uuids):
|
||||
self.expect_uuid_request()
|
||||
|
||||
def expect_save_doc(self, doc, rev=None, **kwargs):
|
||||
if not rev:
|
||||
rev = 1
|
||||
|
||||
self.couchdb.expect_request(
|
||||
method="PUT",
|
||||
path=self.db_path + doc["_id"],
|
||||
body_json=doc,
|
||||
code=201,
|
||||
respond_json={"id": doc["_id"], "rev": self.gen_fake_rev(rev)},
|
||||
**kwargs
|
||||
)
|
||||
|
||||
def test_uses_server_uuids(self):
|
||||
should_use_uuids = []
|
||||
|
||||
for i in xrange(200):
|
||||
uuid = self.pop_uuid()
|
||||
should_use_uuids.append(uuid)
|
||||
|
||||
doc = {
|
||||
"_id": uuid,
|
||||
"time_created": self.callbacks.time_project(i),
|
||||
"time_uploaded": self.callbacks.time_project(i),
|
||||
"data": {
|
||||
"callsign": "PROXYCALL",
|
||||
"test": 123.356
|
||||
},
|
||||
"type": "listener_telemetry"
|
||||
}
|
||||
|
||||
self.expect_save_doc(doc, advance_time_after=1)
|
||||
|
||||
self.couchdb.run()
|
||||
|
||||
for i in xrange(200):
|
||||
doc_id = self.uploader.listener_telemetry({"test": 123.356})
|
||||
assert doc_id == should_use_uuids[i]
|
||||
|
||||
self.couchdb.check()
|
||||
|
||||
def add_sample_listener_docs(self):
|
||||
telemetry_data = {"some_data": 123, "_flag": True}
|
||||
telemetry_doc = {
|
||||
"_id": self.pop_uuid(),
|
||||
"data": copy.deepcopy(telemetry_data),
|
||||
"type": "listener_telemetry",
|
||||
"time_created": self.callbacks.time_project(0),
|
||||
"time_uploaded": self.callbacks.time_project(0)
|
||||
}
|
||||
telemetry_doc["data"]["callsign"] = "PROXYCALL"
|
||||
|
||||
info_data = {"my_radio": "Duga-3", "vehicle": "Tractor"}
|
||||
info_doc = {
|
||||
"_id": self.pop_uuid(),
|
||||
"data": copy.deepcopy(info_data),
|
||||
"type": "listener_info",
|
||||
"time_created": self.callbacks.time_project(0),
|
||||
"time_uploaded": self.callbacks.time_project(0)
|
||||
}
|
||||
info_doc["data"]["callsign"] = "PROXYCALL"
|
||||
|
||||
self.expect_save_doc(telemetry_doc)
|
||||
self.expect_save_doc(info_doc)
|
||||
|
||||
self.couchdb.run()
|
||||
self.sample_telemetry_doc_id = \
|
||||
self.uploader.listener_telemetry(telemetry_data)
|
||||
self.sample_info_doc_id = self.uploader.listener_info(info_data)
|
||||
self.couchdb.check()
|
||||
|
||||
assert self.sample_telemetry_doc_id == telemetry_doc["_id"]
|
||||
assert self.sample_info_doc_id == info_doc["_id"]
|
||||
|
||||
def test_pushes_listener_docs(self):
|
||||
self.add_sample_listener_docs()
|
||||
|
||||
# And now again, but this time, setting time_created.
|
||||
telemetry_data = {
|
||||
"time": {
|
||||
"hour": 12,
|
||||
"minute": 40,
|
||||
"second": 5
|
||||
},
|
||||
"latitude": 35.11,
|
||||
"longitude": 137.567,
|
||||
"altitude": 12
|
||||
}
|
||||
telemetry_doc = {
|
||||
"_id": self.pop_uuid(),
|
||||
"data": copy.deepcopy(telemetry_data),
|
||||
"type": "listener_telemetry",
|
||||
"time_created": 501,
|
||||
"time_uploaded": self.callbacks.time_project(0)
|
||||
}
|
||||
telemetry_doc["data"]["callsign"] = "PROXYCALL"
|
||||
|
||||
info_data = {
|
||||
"name": "Daniel Richman",
|
||||
"location": "Reading, UK",
|
||||
"radio": "Yaesu FT 790R",
|
||||
"antenna": "Whip"
|
||||
}
|
||||
info_doc = {
|
||||
"_id": self.pop_uuid(),
|
||||
"data": copy.deepcopy(info_data),
|
||||
"type": "listener_info",
|
||||
"time_created": 409,
|
||||
"time_uploaded": self.callbacks.time_project(5)
|
||||
}
|
||||
info_doc["data"]["callsign"] = "PROXYCALL"
|
||||
|
||||
self.expect_save_doc(telemetry_doc, advance_time_after=5)
|
||||
self.expect_save_doc(info_doc)
|
||||
|
||||
self.couchdb.run()
|
||||
telemetry_doc_id = \
|
||||
self.uploader.listener_telemetry(telemetry_data, 501)
|
||||
info_doc_id = self.uploader.listener_info(info_data, 409)
|
||||
self.couchdb.check()
|
||||
|
||||
assert telemetry_doc_id == telemetry_doc["_id"]
|
||||
assert info_doc_id == info_doc["_id"]
|
||||
|
||||
ptlm_doc_id = "c0be13b259acfd2fe23cd0d1e70555d6" \
|
||||
"8f83926278b23f5b813bdc75f6b9cdd6"
|
||||
ptlm_string = "asdf blah \x12 binar\x04\x01 asdfasdfsz"
|
||||
ptlm_metadata = {"frequency": 434075000, "misc": "Hi"}
|
||||
ptlm_doc = {
|
||||
"_id": ptlm_doc_id,
|
||||
"data": {
|
||||
"_raw": "YXNkZiBibGFoIBIgYmluYXIEASBhc2RmYXNkZnN6"
|
||||
},
|
||||
"type": "payload_telemetry",
|
||||
"receivers": {
|
||||
"PROXYCALL": {
|
||||
"frequency": 434075000,
|
||||
"misc": "Hi"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
def test_payload_telemetry(self):
|
||||
# WARNING: JsonCPP does not support strings with \0 in the middle of
|
||||
# them, because it does not store the length of the string and instead
|
||||
# later figures it out with strlen. This does not harm the uploader
|
||||
# because our code converts binary data to base64 before giving it
|
||||
# to the json encoder. However, the json stdin proxy call interface
|
||||
# isn't going to work with nulls in it.
|
||||
|
||||
receiver_info = {
|
||||
"time_created": self.callbacks.time_project(0),
|
||||
"time_uploaded": self.callbacks.time_project(0),
|
||||
}
|
||||
|
||||
doc = copy.deepcopy(self.ptlm_doc)
|
||||
doc["receivers"]["PROXYCALL"].update(receiver_info)
|
||||
|
||||
self.expect_save_doc(doc)
|
||||
self.couchdb.run()
|
||||
ret_doc_id = self.uploader.payload_telemetry(self.ptlm_string,
|
||||
self.ptlm_metadata)
|
||||
self.couchdb.check()
|
||||
|
||||
assert ret_doc_id == self.ptlm_doc_id
|
||||
|
||||
def test_adds_latest_listener_doc(self):
|
||||
self.add_sample_listener_docs()
|
||||
|
||||
receiver_info = {
|
||||
"time_created": self.callbacks.time_project(0),
|
||||
"time_uploaded": self.callbacks.time_project(0),
|
||||
"latest_listener_telemetry": self.sample_telemetry_doc_id,
|
||||
"latest_listener_info": self.sample_info_doc_id
|
||||
}
|
||||
|
||||
doc = copy.deepcopy(self.ptlm_doc)
|
||||
doc["receivers"]["PROXYCALL"].update(receiver_info)
|
||||
|
||||
self.expect_save_doc(doc)
|
||||
self.couchdb.run()
|
||||
self.uploader.payload_telemetry(self.ptlm_string, self.ptlm_metadata)
|
||||
self.couchdb.check()
|
||||
|
||||
ptlm_doc_existing = {
|
||||
"_id": ptlm_doc_id,
|
||||
"data": {
|
||||
"_raw": "YXNkZiBibGFoIBIgYmluYXIEASBhc2RmYXNkZnN6",
|
||||
"some_parsed_data": 12345
|
||||
},
|
||||
"type": "payload_telemetry",
|
||||
"receivers": {
|
||||
"SOMEONEELSE": {
|
||||
"time_created": 200,
|
||||
"time_uploaded": 240,
|
||||
"frequency": 434074000,
|
||||
"asdf": "World"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
def test_ptlm_merges_payload_conflicts(self):
|
||||
receiver_info = {
|
||||
"time_created": self.callbacks.time_project(0),
|
||||
"time_uploaded": self.callbacks.time_project(0),
|
||||
}
|
||||
|
||||
doc = copy.deepcopy(self.ptlm_doc)
|
||||
doc["receivers"]["PROXYCALL"].update(receiver_info)
|
||||
|
||||
self.couchdb.expect_request(
|
||||
method="PUT",
|
||||
path=self.db_path + doc["_id"],
|
||||
body_json=doc,
|
||||
code=409,
|
||||
respond_json={"error": "conflict"},
|
||||
advance_time_after=5
|
||||
)
|
||||
|
||||
self.couchdb.expect_request(
|
||||
path=self.db_path + self.ptlm_doc_id,
|
||||
code=200,
|
||||
respond_json=self.ptlm_doc_existing,
|
||||
advance_time_after=5,
|
||||
)
|
||||
|
||||
doc_merged = copy.deepcopy(self.ptlm_doc_existing)
|
||||
doc_merged["receivers"]["PROXYCALL"] = \
|
||||
copy.deepcopy(self.ptlm_doc["receivers"]["PROXYCALL"])
|
||||
receiver_info = copy.deepcopy(receiver_info)
|
||||
receiver_info["time_uploaded"] = self.callbacks.time_project(10)
|
||||
doc_merged["receivers"]["PROXYCALL"].update(receiver_info)
|
||||
|
||||
self.expect_save_doc(doc_merged)
|
||||
|
||||
self.couchdb.run()
|
||||
self.uploader.payload_telemetry(self.ptlm_string, self.ptlm_metadata)
|
||||
self.couchdb.check()
|
||||
|
||||
def test_ptlm_refuses_to_merge_collision(self):
|
||||
receiver_info = {
|
||||
"time_created": self.callbacks.time_project(0),
|
||||
"time_uploaded": self.callbacks.time_project(0),
|
||||
}
|
||||
|
||||
doc = copy.deepcopy(self.ptlm_doc)
|
||||
doc["receivers"]["PROXYCALL"].update(receiver_info)
|
||||
del doc["receivers"]["PROXYCALL"]["frequency"]
|
||||
del doc["receivers"]["PROXYCALL"]["misc"]
|
||||
|
||||
self.couchdb.expect_request(
|
||||
method="PUT",
|
||||
path=self.db_path + doc["_id"],
|
||||
body_json=doc,
|
||||
code=409,
|
||||
respond_json={"error": "conflict"},
|
||||
advance_time_after=5
|
||||
)
|
||||
|
||||
doc_conflict = copy.deepcopy(self.ptlm_doc_existing)
|
||||
doc_conflict["data"]["_raw"] = "cGluZWFwcGxlcw=="
|
||||
|
||||
self.couchdb.expect_request(
|
||||
path=self.db_path + self.ptlm_doc_id,
|
||||
code=200,
|
||||
respond_json=doc_conflict
|
||||
)
|
||||
|
||||
self.couchdb.run()
|
||||
|
||||
try:
|
||||
self.uploader.payload_telemetry(self.ptlm_string)
|
||||
except ProxyException, e:
|
||||
if e.name == "runtime_error" and \
|
||||
e.what == "habitat::CollisionError":
|
||||
pass
|
||||
else:
|
||||
raise
|
||||
else:
|
||||
raise AssertionError("Did not raise CollisionError")
|
||||
|
||||
self.couchdb.check()
|
||||
|
||||
def add_mock_conflicts(self, n):
|
||||
receiver_info = {
|
||||
"time_created": self.callbacks.time_project(0),
|
||||
"time_uploaded": self.callbacks.time_project(0),
|
||||
}
|
||||
|
||||
doc = copy.deepcopy(self.ptlm_doc)
|
||||
doc["receivers"]["PROXYCALL"].update(receiver_info)
|
||||
|
||||
self.couchdb.expect_request(
|
||||
method="PUT",
|
||||
path=self.db_path + self.ptlm_doc_id,
|
||||
body_json=doc,
|
||||
code=409,
|
||||
respond_json={"error": "conflict"}
|
||||
)
|
||||
|
||||
doc_existing = copy.deepcopy(self.ptlm_doc_existing)
|
||||
|
||||
doc_merged = copy.deepcopy(self.ptlm_doc_existing)
|
||||
doc_merged["receivers"]["PROXYCALL"] = \
|
||||
copy.deepcopy(self.ptlm_doc["receivers"]["PROXYCALL"])
|
||||
doc_merged["receivers"]["PROXYCALL"].update(receiver_info)
|
||||
|
||||
for i in xrange(n):
|
||||
self.couchdb.expect_request(
|
||||
path=self.db_path + self.ptlm_doc_id,
|
||||
code=200,
|
||||
respond_json=doc_existing,
|
||||
)
|
||||
|
||||
self.couchdb.expect_request(
|
||||
method="PUT",
|
||||
path=self.db_path + self.ptlm_doc_id,
|
||||
body_json=doc_merged,
|
||||
code=409,
|
||||
respond_json={"error": "conflict"},
|
||||
advance_time_after=1
|
||||
)
|
||||
|
||||
doc_existing = copy.deepcopy(doc_existing)
|
||||
doc_merged = copy.deepcopy(doc_merged)
|
||||
|
||||
new_call = "listener_{0}".format(i)
|
||||
new_info = {"time_created": 600 + i, "time_uploaded": 641 + i}
|
||||
|
||||
doc_existing["receivers"][new_call] = new_info
|
||||
doc_merged["receivers"][new_call] = new_info
|
||||
|
||||
doc_merged["receivers"]["PROXYCALL"]["time_uploaded"] = \
|
||||
self.callbacks.time_project(i + 1)
|
||||
|
||||
return (doc_existing, doc_merged)
|
||||
|
||||
def test_merges_multiple_conflicts(self):
|
||||
(final_doc_existing, final_doc_merged) = self.add_mock_conflicts(15)
|
||||
|
||||
self.couchdb.expect_request(
|
||||
path=self.db_path + self.ptlm_doc_id,
|
||||
code=200,
|
||||
respond_json=final_doc_existing,
|
||||
)
|
||||
|
||||
self.expect_save_doc(final_doc_merged)
|
||||
|
||||
self.couchdb.run()
|
||||
self.uploader.payload_telemetry(self.ptlm_string, self.ptlm_metadata)
|
||||
self.couchdb.check()
|
||||
|
||||
def test_gives_up_after_many_conflicts(self):
|
||||
self.add_mock_conflicts(20)
|
||||
self.couchdb.run()
|
||||
|
||||
try:
|
||||
self.uploader.payload_telemetry(self.ptlm_string,
|
||||
self.ptlm_metadata)
|
||||
except ProxyException, e:
|
||||
if e.name == "runtime_error" and \
|
||||
e.what == "habitat::UnmergeableError":
|
||||
pass
|
||||
else:
|
||||
raise
|
||||
else:
|
||||
raise AssertionError("Did not raise UnmergeableError")
|
||||
|
||||
def test_flights(self):
|
||||
flights= [{"_id": "flight_{0}".format(i), "a flight": i}
|
||||
for i in xrange(100)]
|
||||
rows = [{"id": doc["_id"], "key": None, "value": None, "doc": doc}
|
||||
for doc in flights]
|
||||
fake_view_response = \
|
||||
{"total_rows": len(rows), "offset": 0, "rows": rows}
|
||||
|
||||
# cURL is a little overzealous with its escape(): _ is replaced
|
||||
# with %5F. This should be fine
|
||||
|
||||
self.callbacks.advance_time(1925)
|
||||
view_time = self.callbacks.time_project(1925)
|
||||
options = "include%5Fdocs=true&startkey=" + str(view_time)
|
||||
|
||||
self.couchdb.expect_request(
|
||||
path=self.db_path + "_design/uploader%5Fv1/_view/flights?" + options,
|
||||
code=200,
|
||||
respond_json=copy.deepcopy(fake_view_response)
|
||||
)
|
||||
self.couchdb.run()
|
||||
|
||||
result = self.uploader.flights()
|
||||
assert result == flights
|
||||
|
||||
class TestCPPConnectorThreaded(TestCPPConnector):
|
||||
command = "tests/cpp_connector_threaded"
|
||||
|
||||
def test_queues_things(self):
|
||||
telemetry_data = {"this was queued": True}
|
||||
telemetry_doc = {
|
||||
"_id": self.pop_uuid(),
|
||||
"data": copy.deepcopy(telemetry_data),
|
||||
"type": "listener_telemetry",
|
||||
"time_created": self.callbacks.time_project(0),
|
||||
"time_uploaded": self.callbacks.time_project(0)
|
||||
}
|
||||
telemetry_doc["data"]["callsign"] = "PROXYCALL"
|
||||
|
||||
info_data = {"5": "this was the second item in the queue"}
|
||||
info_doc = {
|
||||
"_id": self.pop_uuid(),
|
||||
"data": copy.deepcopy(info_data),
|
||||
"type": "listener_info",
|
||||
"time_created": self.callbacks.time_project(0),
|
||||
"time_uploaded": self.callbacks.time_project(0)
|
||||
}
|
||||
info_doc["data"]["callsign"] = "PROXYCALL"
|
||||
|
||||
delay_one = threading.Event()
|
||||
wait_one = threading.Event()
|
||||
delay_two = threading.Event()
|
||||
wait_two = threading.Event()
|
||||
|
||||
self.expect_save_doc(telemetry_doc, delay=delay_one, wait=wait_one)
|
||||
self.expect_save_doc(info_doc, delay=delay_two, wait=wait_two)
|
||||
|
||||
self.couchdb.run()
|
||||
|
||||
self.run_unblocked(self.uploader.listener_telemetry, telemetry_data)
|
||||
self.run_unblocked(self.uploader.listener_info, info_data)
|
||||
|
||||
# The complexity of doing this properly justifies this evil hack...
|
||||
# right?
|
||||
while not wait_one.is_set():
|
||||
wait_one.wait(0.1)
|
||||
self.run_unblocked(self.uploader.complete)
|
||||
|
||||
delay_one.set()
|
||||
assert self.uploader.complete() == telemetry_doc["_id"]
|
||||
|
||||
while not wait_two.is_set():
|
||||
wait_two.wait(0.01)
|
||||
self.run_unblocked(self.uploader.complete)
|
||||
|
||||
delay_two.set()
|
||||
assert self.uploader.complete() == info_doc["_id"]
|
||||
|
||||
self.couchdb.check()
|
||||
|
||||
def run_unblocked(self, func, *args, **kwargs):
|
||||
self.uploader.unblock()
|
||||
|
||||
try:
|
||||
return func(*args, **kwargs)
|
||||
except IOError as e:
|
||||
if e.errno != errno.EAGAIN:
|
||||
raise
|
||||
else:
|
||||
raise AssertionError("expected IOError(EAGAIN)")
|
||||
|
||||
self.uploader.block()
|
||||
|
||||
def test_changes_settings(self):
|
||||
self.uploader.re_init("NEWCALL", self.couchdb.url)
|
||||
|
||||
receiver_info = {
|
||||
"time_created": self.callbacks.time_project(0),
|
||||
"time_uploaded": self.callbacks.time_project(0),
|
||||
}
|
||||
|
||||
doc = copy.deepcopy(self.ptlm_doc)
|
||||
doc["receivers"]["PROXYCALL"].update(receiver_info)
|
||||
doc["receivers"]["NEWCALL"] = doc["receivers"]["PROXYCALL"]
|
||||
del doc["receivers"]["PROXYCALL"]
|
||||
|
||||
self.expect_save_doc(doc)
|
||||
self.couchdb.run()
|
||||
ret_doc_id = self.uploader.payload_telemetry(self.ptlm_string,
|
||||
self.ptlm_metadata)
|
||||
self.couchdb.check()
|
||||
|
||||
assert ret_doc_id == self.ptlm_doc_id
|
||||
|
||||
def test_reset(self):
|
||||
self.uploader.re_init("NEWCALL", self.couchdb.url)
|
||||
self.uploader.reset()
|
||||
|
||||
self.couchdb.run() # expect nothing.
|
||||
|
||||
try:
|
||||
self.uploader.payload_telemetry("asdf", {})
|
||||
except ProxyException as e:
|
||||
assert "not initialised" in str(e)
|
||||
else:
|
||||
raise AssertionError("not initialised was not thrown")
|
||||
|
||||
self.couchdb.check()
|
||||
|
|
@ -0,0 +1,372 @@
|
|||
/* Copyright 2011 (C) Daniel Richman. License: GNU GPL 3; see LICENSE. */
|
||||
|
||||
#include <iostream>
|
||||
#include <memory>
|
||||
#include <stdexcept>
|
||||
#include <ctime>
|
||||
|
||||
#include "EZ.h"
|
||||
#include "Uploader.h"
|
||||
|
||||
#ifdef THREADED
|
||||
#include "UploaderThread.h"
|
||||
#endif
|
||||
|
||||
using namespace std;
|
||||
|
||||
template <typename T>
|
||||
class SafeValue
|
||||
{
|
||||
EZ::Mutex mutex;
|
||||
T value;
|
||||
|
||||
public:
|
||||
SafeValue(T init) : value(init) {};
|
||||
~SafeValue() {};
|
||||
bool get() { EZ::MutexLock lock(mutex); return value; }
|
||||
void set(T v) { EZ::MutexLock lock(mutex); value = v; }
|
||||
};
|
||||
|
||||
static Json::Value proxy_callback(const string &name, const Json::Value &args);
|
||||
static Json::Value repackage_flights(const vector<Json::Value> &flights);
|
||||
static void report_result(const Json::Value &arg1,
|
||||
const Json::Value &arg2=Json::Value::null,
|
||||
const Json::Value &arg3=Json::Value::null);
|
||||
|
||||
#ifndef THREADED
|
||||
typedef habitat::Uploader TestSubject;
|
||||
typedef string r_string;
|
||||
typedef Json::Value r_json;
|
||||
static TestSubject *proxy_constructor(Json::Value command);
|
||||
#else
|
||||
class TestUploaderThread : public habitat::UploaderThread
|
||||
{
|
||||
void log(const string &message) { report_result("log", message); };
|
||||
|
||||
void saved_id(const string &type, const string &id)
|
||||
{ report_result("return", id); };
|
||||
|
||||
void initialised() { report_result("return"); };
|
||||
|
||||
void reset_done() { report_result("return"); };
|
||||
|
||||
void caught_exception(const runtime_error &error)
|
||||
{ report_result("error", "runtime_error", error.what()); }
|
||||
|
||||
void caught_exception(const invalid_argument &error)
|
||||
{ report_result("error", "invalid_argument", error.what()); }
|
||||
|
||||
void got_flights(const vector<Json::Value> &flights)
|
||||
{ report_result("return", repackage_flights(flights)); }
|
||||
};
|
||||
|
||||
typedef TestUploaderThread TestSubject;
|
||||
typedef void r_string;
|
||||
typedef void r_json;
|
||||
static void proxy_constructor(TestSubject *u, Json::Value command);
|
||||
static void proxy_reset(TestSubject *u);
|
||||
#endif
|
||||
|
||||
static r_string proxy_listener_info(TestSubject *u, Json::Value command);
|
||||
static r_string proxy_listener_telemetry(TestSubject *u, Json::Value command);
|
||||
static r_string proxy_payload_telemetry(TestSubject *u, Json::Value command);
|
||||
static r_json proxy_flights(TestSubject *u);
|
||||
|
||||
static EZ::cURLGlobal cgl;
|
||||
static EZ::Mutex cout_lock;
|
||||
static SafeValue<bool> enable_callbacks(false);
|
||||
static SafeValue<int> last_time(10000);
|
||||
|
||||
#ifdef THREADED
|
||||
static EZ::Queue<Json::Value> callback_responses;
|
||||
#endif
|
||||
|
||||
int main(int argc, char **argv)
|
||||
{
|
||||
#ifndef THREADED
|
||||
auto_ptr<habitat::Uploader> u;
|
||||
#else
|
||||
enable_callbacks.set(true);
|
||||
TestSubject thread;
|
||||
thread.start();
|
||||
#endif
|
||||
|
||||
for (;;)
|
||||
{
|
||||
char line[1024];
|
||||
cin.getline(line, 1024);
|
||||
|
||||
if (line[0] == '\0')
|
||||
{
|
||||
#ifdef THREADED
|
||||
enable_callbacks.set(false);
|
||||
#endif
|
||||
break;
|
||||
}
|
||||
|
||||
Json::Reader reader;
|
||||
Json::Value command;
|
||||
|
||||
if (!reader.parse(line, command, false))
|
||||
throw runtime_error("JSON parsing failed");
|
||||
|
||||
if (!command.isArray() || !command[0u].isString())
|
||||
throw runtime_error("Invalid JSON input");
|
||||
|
||||
string command_name = command[0u].asString();
|
||||
|
||||
#ifndef THREADED
|
||||
if (!u.get() && command_name != "init")
|
||||
throw runtime_error("You must initialise it first");
|
||||
|
||||
enable_callbacks.set(true);
|
||||
|
||||
try
|
||||
{
|
||||
Json::Value return_value;
|
||||
|
||||
if (command_name == "init")
|
||||
u.reset(proxy_constructor(command));
|
||||
else if (command_name == "listener_info")
|
||||
return_value = proxy_listener_info(u.get(), command);
|
||||
else if (command_name == "listener_telemetry")
|
||||
return_value = proxy_listener_telemetry(u.get(), command);
|
||||
else if (command_name == "payload_telemetry")
|
||||
return_value = proxy_payload_telemetry(u.get(), command);
|
||||
else if (command_name == "flights")
|
||||
return_value = proxy_flights(u.get());
|
||||
else
|
||||
throw runtime_error("invalid command name");
|
||||
|
||||
if (command_name != "init")
|
||||
report_result("return", return_value);
|
||||
else
|
||||
report_result("return");
|
||||
}
|
||||
catch (runtime_error e)
|
||||
{
|
||||
if (e.what() == string("invalid command name"))
|
||||
throw;
|
||||
|
||||
report_result("error", "runtime_error", e.what());
|
||||
}
|
||||
catch (invalid_argument e)
|
||||
{
|
||||
report_result("error", "invalid_argument", e.what());
|
||||
}
|
||||
catch (...)
|
||||
{
|
||||
report_result("error", "unknown_error");
|
||||
}
|
||||
|
||||
enable_callbacks.set(false);
|
||||
#else
|
||||
if (command_name == "init")
|
||||
proxy_constructor(&thread, command);
|
||||
else if (command_name == "reset")
|
||||
proxy_reset(&thread);
|
||||
else if (command_name == "listener_info")
|
||||
proxy_listener_info(&thread, command);
|
||||
else if (command_name == "listener_telemetry")
|
||||
proxy_listener_telemetry(&thread, command);
|
||||
else if (command_name == "payload_telemetry")
|
||||
proxy_payload_telemetry(&thread, command);
|
||||
else if (command_name == "flights")
|
||||
proxy_flights(&thread);
|
||||
else if (command_name == "return")
|
||||
callback_responses.put(command);
|
||||
#endif
|
||||
}
|
||||
|
||||
#ifdef THREADED
|
||||
thread.shutdown();
|
||||
thread.join();
|
||||
#endif
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
time_t time(time_t *t) throw()
|
||||
{
|
||||
time_t value;
|
||||
|
||||
if (!enable_callbacks.get())
|
||||
{
|
||||
value = last_time.get();
|
||||
}
|
||||
else
|
||||
{
|
||||
Json::Value result = proxy_callback("time", Json::Value::null);
|
||||
|
||||
if (!result.isInt())
|
||||
throw runtime_error("invalid callback response");
|
||||
|
||||
value = result.asInt();
|
||||
}
|
||||
|
||||
last_time.set(value);
|
||||
|
||||
if (t)
|
||||
*t = value;
|
||||
|
||||
return value;
|
||||
}
|
||||
|
||||
static Json::Value proxy_callback(const string &name, const Json::Value &args)
|
||||
{
|
||||
report_result("callback", name, args);
|
||||
|
||||
#ifndef THREADED
|
||||
char line[1024];
|
||||
cin.getline(line, 1024);
|
||||
|
||||
Json::Reader reader;
|
||||
Json::Value response;
|
||||
|
||||
if (!reader.parse(line, response, false))
|
||||
throw runtime_error("JSON parsing failed");
|
||||
|
||||
if (!response.isArray() || !response[0u].isString())
|
||||
throw runtime_error("Invalid callback response");
|
||||
|
||||
if (response[0u].asString() != "return")
|
||||
throw runtime_error("Callback failed");
|
||||
#else
|
||||
Json::Value response = callback_responses.get();
|
||||
#endif
|
||||
|
||||
return response[1u];
|
||||
}
|
||||
|
||||
#ifndef THREADED
|
||||
static habitat::Uploader *proxy_constructor(Json::Value command)
|
||||
#else
|
||||
static void proxy_constructor(TestSubject *u, Json::Value command)
|
||||
#endif
|
||||
{
|
||||
const Json::Value &callsign = command[1u];
|
||||
const Json::Value &couch_uri = command[2u];
|
||||
const Json::Value &couch_db = command[3u];
|
||||
const Json::Value &max_merge_attempts = command[4u];
|
||||
|
||||
/* .isString is checked when .asString is used. */
|
||||
if (!max_merge_attempts.isNull() && !max_merge_attempts.isInt())
|
||||
throw invalid_argument("max_merge_attempts");
|
||||
|
||||
#ifndef THREADED
|
||||
#define construct_it(...) do { return new TestSubject(__VA_ARGS__); } while (0)
|
||||
#else
|
||||
#define construct_it(...) do { u->settings(__VA_ARGS__); } while (0)
|
||||
#endif
|
||||
|
||||
if (max_merge_attempts.isNull() && couch_db.isNull() &&
|
||||
couch_uri.isNull())
|
||||
{
|
||||
construct_it(callsign.asString());
|
||||
}
|
||||
else if (max_merge_attempts.isNull() && couch_db.isNull())
|
||||
{
|
||||
construct_it(callsign.asString(), couch_uri.asString());
|
||||
}
|
||||
else if (max_merge_attempts.isNull())
|
||||
{
|
||||
construct_it(callsign.asString(), couch_uri.asString(),
|
||||
couch_db.asString());
|
||||
}
|
||||
else
|
||||
{
|
||||
construct_it(callsign.asString(), couch_uri.asString(),
|
||||
couch_db.asString(), max_merge_attempts.asInt());
|
||||
}
|
||||
|
||||
#undef construct_it
|
||||
}
|
||||
|
||||
#ifdef THREADED
|
||||
static void proxy_reset(TestSubject *u)
|
||||
{
|
||||
u->reset();
|
||||
}
|
||||
#endif
|
||||
|
||||
static r_string proxy_listener_info(TestSubject *u, Json::Value command)
|
||||
{
|
||||
const Json::Value &data = command[1u];
|
||||
const Json::Value &tc = command[2u];
|
||||
|
||||
if (tc.isNull())
|
||||
return u->listener_info(data);
|
||||
else
|
||||
return u->listener_info(data, tc.asInt());
|
||||
}
|
||||
|
||||
static r_string proxy_listener_telemetry(TestSubject *u, Json::Value command)
|
||||
{
|
||||
const Json::Value &data = command[1u];
|
||||
const Json::Value &tc = command[2u];
|
||||
|
||||
if (tc.isNull())
|
||||
return u->listener_telemetry(data);
|
||||
else
|
||||
return u->listener_telemetry(data, tc.asInt());
|
||||
}
|
||||
|
||||
static r_string proxy_payload_telemetry(TestSubject *u, Json::Value command)
|
||||
{
|
||||
const Json::Value &data = command[1u];
|
||||
const Json::Value &metadata = command[2u];
|
||||
const Json::Value &tc = command[3u];
|
||||
|
||||
if (tc.isNull() && metadata.isNull())
|
||||
return u->payload_telemetry(data.asString());
|
||||
else if (tc.isNull())
|
||||
return u->payload_telemetry(data.asString(), metadata);
|
||||
else
|
||||
return u->payload_telemetry(data.asString(), metadata, tc.asInt());
|
||||
}
|
||||
|
||||
static r_json proxy_flights(TestSubject *u)
|
||||
{
|
||||
#ifndef THREADED
|
||||
vector<Json::Value> *result = u->flights();
|
||||
auto_ptr< vector<Json::Value> > destroyer(result);
|
||||
return repackage_flights(*result);
|
||||
#else
|
||||
u->flights();
|
||||
#endif
|
||||
}
|
||||
|
||||
static Json::Value repackage_flights(const vector<Json::Value> &flights)
|
||||
{
|
||||
Json::Value list(Json::arrayValue);
|
||||
vector<Json::Value>::const_iterator it;
|
||||
for (it = flights.begin(); it != flights.end(); it++)
|
||||
list.append(*it);
|
||||
return list;
|
||||
}
|
||||
|
||||
static void report_result(const Json::Value &arg1, const Json::Value &arg2,
|
||||
const Json::Value &arg3)
|
||||
{
|
||||
Json::Value report(Json::arrayValue);
|
||||
|
||||
report.append(arg1);
|
||||
|
||||
if (!arg2.isNull())
|
||||
{
|
||||
report.append(arg2);
|
||||
|
||||
if (!arg3.isNull())
|
||||
{
|
||||
report.append(arg3);
|
||||
}
|
||||
}
|
||||
|
||||
Json::FastWriter writer;
|
||||
|
||||
{
|
||||
EZ::MutexLock lock(cout_lock);
|
||||
cout << writer.write(report);
|
||||
cout.flush();
|
||||
}
|
||||
}
|
||||
Ładowanie…
Reference in New Issue