Reading and processing information from a database » History » Version 2
Rafael Bailon-Ruiz, 2020-11-12 18:15
1 | 1 | Rafael Bailon-Ruiz | h1. Reading and processing information from a database |
---|---|---|---|
2 | |||
3 | The purpose of this page is to show you how to use the CAMS database and how to process data inside without modifying stored data. |
||
4 | |||
5 | 2 | Rafael Bailon-Ruiz | {{toc}} |
6 | |||
7 | 1 | Rafael Bailon-Ruiz | h2. The database |
8 | |||
9 | The databases objects here are SpatializedDatabase and its subclass, NephelaeDataServer. While being both usable, only the NephelaeDataServer is being used to register data and play simulations / real-flights. The reason is because this version is tailored to the CAMS needs (notification methods, uav ids, variable names...). |
||
10 | Because of these reasons, this paragraph will only talk about the NephelaeDataServer. |
||
11 | The NephelaeDataServer uses SpbEntry objects to register data. These objects have a Position to know where the data has been found, aswell as a list of tags to know what type of data is inside. The data attribute of SpbEntry are either AircraftStatus object or SensorSample during simulations. |
||
12 | You can retrieve data from the database using the __getitem__() function or the find_entries() (see the getitem function as a sugar syntax of the find entries function). This will return a list of SpbEntries. |
||
13 | Usage exemple : |
||
14 | <pre><code class=python>db = NephelaeDataServer.load('path/to/some/file') |
||
15 | db.find_entries(keys=keys, tags=['RCT']) # Finding all the SpbEntry related to the RCT</code></pre> |
||
16 | NephelaeDataServer comes also with notification methods, these methods are add_gps(), add_status() and add_sample(). |
||
17 | These methods are always called when an Aircraft is sending data to the database, notifying these methods. |
||
18 | |||
19 | h2. Process data |
||
20 | |||
21 | Since the data registered in the database is raw data, we use objects that processes copies of registered data. |
||
22 | Why registering raw data ? Because of reuse ability. This way we can take this data to reproduce an experiment and try differently. |
||
23 | To process copies of registered data, we use a dataview. It's a sort of "view" that process copies of registered data. There is a lot of dataviews types, each for a specific usage. |
||
24 | A non-exhaustive list of usages : |
||
25 | |||
26 | * Fetching from the database |
||
27 | |||
28 | * Process a certain type of sample |
||
29 | |||
30 | * Fetching status of an UAV |
||
31 | |||
32 | * Reorganize data |
||
33 | A dataview can generally have multiple parents and children dataviews. Fetching data from a dataview will return the result of all the parent dataviews, applying the different data process. Once finished, the dataview will apply its own data process and return it. |
||
34 | 2 | Rafael Bailon-Ruiz | |
35 | h2. New database format |
||
36 | |||
37 | h3. Data model |
||
38 | |||
39 | The CAMS database model is based on the OGR data model and the OGC OpenPackage specification. |
||
40 | |||
41 | * Dataset (= OGR Dataset) |
||
42 | ** Collection (= OGR Layer) |
||
43 | *** Record (= OGR Feature) |
||
44 | **** Mandatory fields: |
||
45 | ***** geom [automatic, the location of the record] (= OGR Point 3D Geometry) |
||
46 | ***** t : (=OGR OFTDateTime) |
||
47 | ***** producer : (=ogr OFTString with 80 chars) |
||
48 | **** Optional fields: |
||
49 | ***** data associated with the records. They can be of integer, real and text types. |