Creating new projects, flights, and data products¶

This guide will walk you through the steps for creating new data on a Data to Science (D2S) instance.

To get started, you will need to import the Workspace modules.

In [ ]:

# Uncomment and run the following line if working out of Google Colab
# !pip install d2spy

# Uncomment and run the following line if working out of Google Colab # !pip install d2spy

In [ ]:

from datetime import date

from d2spy.workspace import Workspace

from datetime import date from d2spy.workspace import Workspace

All of your D2S data can be accessed through a D2S "workspace." The Workspace module's connect method can be used to login to a D2S instance and connect to your workspace in one go. Behind the scenes, the Auth module will be used to handle authenticating with D2S and requesting an authorization token. You will need to provide connect with the URL to your D2S instance and enter your password when prompted. Once connected to your workspace, you will be able to start creating projects within it.

In [ ]:

# Example of connecting to a workspace for a local D2S instance
workspace = Workspace.connect("http://localhost:8000", "yourD2Semail@example.com")

# Example of connecting to a workspace for a local D2S instance workspace = Workspace.connect("http://localhost:8000", "yourD2Semail@example.com")

Creating a new project¶

To create a new project in our workspace, we can use the Workspace module's add_project method as follows:

In [ ]:

# At minimum, must provide title, description, and location (dict in GeoJSON Feature format)
project = workspace.add_project(
    title="Project title",
    description="Project for testing d2spy package.",
    location={
        "type": "Feature",
        "geometry": {
            "type": "Polygon",
            "coordinates": [
                [
                  [-86.944981783977838, 41.444435853085622],
                  [-86.943319754949272, 41.444435046238446],
                  [-86.94332056379109, 41.443505552529658],
                  [-86.944982569102066, 41.443506359350643],
                  [-86.944981783977838, 41.444435853085622]
                ]
            ]
        },
        "properties": {
            "prop0": "value0"
        }
    },
    start_date=date(2024, 4, 15), # Optional
    end_date=date(2024, 9, 15) # Optional
)
print(project)

# At minimum, must provide title, description, and location (dict in GeoJSON Feature format) project = workspace.add_project( title="Project title", description="Project for testing d2spy package.", location={ "type": "Feature", "geometry": { "type": "Polygon", "coordinates": [ [ [-86.944981783977838, 41.444435853085622], [-86.943319754949272, 41.444435046238446], [-86.94332056379109, 41.443505552529658], [-86.944982569102066, 41.443506359350643], [-86.944981783977838, 41.444435853085622] ] ] }, "properties": { "prop0": "value0" } }, start_date=date(2024, 4, 15), # Optional end_date=date(2024, 9, 15) # Optional ) print(project)

The Workspace module's add_project returns a Project instance we can use to create new flights in the project, retrieve existing flights, and update the initially provided project details.

For example, we can use the Project update method to change the project title.

In [ ]:

print(f"Original title: {project.title}")
project.update(title="My First D2S Project")
print(f"Updated title: {project.title}")

print(f"Original title: {project.title}") project.update(title="My First D2S Project") print(f"Updated title: {project.title}")

Adding a flight to a project¶

Since this is a new project, we do not have any available flights yet. We can add a flight using the Project module's add_flight method.

Important

The sensor and platform attributes have some limits on what values will be accepted. Sensor will only accept the values: "RGB", "Multispectral", "LiDAR", and "Other". Platform expects the values "Phantom_4", "M300", or "M350". It can be provided with an alternative value if none of these options are suitable.

Refer to the documentation for more details.

In [ ]:

# Example flight form
flight = project.add_flight(
    acquisition_date=date(2023, 6, 3),  # string in YYYY-MM-DD format also acceptable
    altitude=40,
    side_overlap=85,
    forward_overlap=85,
    sensor="RGB",
    platform="M350"
)
print(flight)

# Example flight form flight = project.add_flight( acquisition_date=date(2023, 6, 3), # string in YYYY-MM-DD format also acceptable altitude=40, side_overlap=85, forward_overlap=85, sensor="RGB", platform="M350" ) print(flight)

Uploading a data product to a flight¶

Now that a flight has been created, you can start uploading data products to the flight. Note, the data product will not be immediately available after the upload completes. When D2S receives a new GeoTIFF, it will check if it is in a cloud-optimized format, and if not, perform this conversion. This process may take several minutes to complete.

The Flight module's add_data_product method will be used to upload a GeoTIFF located on the local machine. You will need to have a GeoTIFF and know its filepath for this next cell.

Important

The data_type has some limits on what values will be accepted. data_type expects the values "dsm", "point_cloud", or "ortho". It can be provided with an alternative value if none of these options are suitable.

Refer to the documentation for more details.

In [ ]:

my_geotiff = "/full/path/to/my/data_product.tif"
flight.add_data_product(
    filepath=my_geotiff,
    data_type="dsm"
)

my_geotiff = "/full/path/to/my/data_product.tif" flight.add_data_product( filepath=my_geotiff, data_type="dsm" )

When D2S has finished processing the upload, the Flight module's get_data_products method can be used to find the uploaded data product and its static URL.

In [ ]:

data_products = flight.get_data_products()
try:
    print(data_products[0].url)
except IndexError:
    print("Data product still processing or an unexpected error has occurred")

data_products = flight.get_data_products() try: print(data_products[0].url) except IndexError: print("Data product still processing or an unexpected error has occurred")

Uploading raw data to a flight¶

Zipped raw data can also be uploaded to a flight using d2spy. In the next cell, the Flight module's add_raw_data method will be used to upload a zipped folder containing raw images located on the local machine.

In [ ]:

my_rawdata = "/full/path/to/my/raw_data.zip"
flight.add_raw_data(filepath=my_rawdata)

my_rawdata = "/full/path/to/my/raw_data.zip" flight.add_raw_data(filepath=my_rawdata)

When D2S has finished processing the upload, the Flight module's get_raw_data method can be used to find the uploaded raw data zip and its static URL.

In [ ]:

raw_data = flight.get_raw_data()
try:
    print(raw_data[0].url)
except IndexError:
    print("Raw data is still processing or an unexpected error has occurred")

raw_data = flight.get_raw_data() try: print(raw_data[0].url) except IndexError: print("Raw data is still processing or an unexpected error has occurred")

Add vector map layers to a project¶

Vector data in the GeoJSON Feature Collection format can be added to a project using a Project's add_map_layer method. The method's layer_name parameter can be used to name the map layer.

In [ ]:

layer_name = "My Map Layer"
feature_collection = {
    "type": "FeatureCollection",
    "features": [
        {
            "type": "Feature",
            "properties": {"row": 1, "col": 1},
            "geometry": {
                "type": "Polygon",
                "coordinates": [
                    [
                        [-86.944517485972483, 41.444077836565455, 0.0],
                        [-86.94450551488066, 41.444077830791521, 0.0],
                        [-86.94450552255509, 41.444068823253602, 0.0],
                        [-86.94451749364525, 41.444068829027536, 0.0],
                        [-86.944517485972483, 41.444077836565455, 0.0],
                    ]
                ],
            },
        },
        {
            "type": "Feature",
            "properties": {"row": 1, "col": 2},
            "geometry": {
                "type": "Polygon",
                "coordinates": [
                    [
                        [-86.944493543788852, 41.444077825016343, 0.0],
                        [-86.944481572697043, 41.444077819239929, 0.0],
                        [-86.94448158037477, 41.44406881170201, 0.0],
                        [-86.94449355146493, 41.444068817478424, 0.0],
                        [-86.944493543788852, 41.444077825016343, 0.0],
                    ]
                ],
            },
        },
    ],
}
# Add the map layer to the project
map_layer_feature_collection = project.add_map_layer(feature_collection=feature_collection, layer_name=layer_name)

# The returned feature collection will contain additional D2S metadata such as the URL for a preview image of the map layer
print(map_layer_feature_collection["metadata"]["preview_url"])

layer_name = "My Map Layer" feature_collection = { "type": "FeatureCollection", "features": [ { "type": "Feature", "properties": {"row": 1, "col": 1}, "geometry": { "type": "Polygon", "coordinates": [ [ [-86.944517485972483, 41.444077836565455, 0.0], [-86.94450551488066, 41.444077830791521, 0.0], [-86.94450552255509, 41.444068823253602, 0.0], [-86.94451749364525, 41.444068829027536, 0.0], [-86.944517485972483, 41.444077836565455, 0.0], ] ], }, }, { "type": "Feature", "properties": {"row": 1, "col": 2}, "geometry": { "type": "Polygon", "coordinates": [ [ [-86.944493543788852, 41.444077825016343, 0.0], [-86.944481572697043, 41.444077819239929, 0.0], [-86.94448158037477, 41.44406881170201, 0.0], [-86.94449355146493, 41.444068817478424, 0.0], [-86.944493543788852, 41.444077825016343, 0.0], ] ], }, }, ], } # Add the map layer to the project map_layer_feature_collection = project.add_map_layer(feature_collection=feature_collection, layer_name=layer_name) # The returned feature collection will contain additional D2S metadata such as the URL for a preview image of the map layer print(map_layer_feature_collection["metadata"]["preview_url"])

These are all the methods you need to know for viewing what data your account can access on a D2S instance. Once finished exploring your data, you can revoke your authorization session by logging out.

In [ ]:

# Removes access token from future requests
workspace.logout()

# Removes access token from future requests workspace.logout()