gbif_insert

Repo Github

gbif_insert is a Python tool used to download occurrences data from www.gbif.org and inject it into atlas postgres database.

Installation

New installation

pip3 install git+https://github.com/BiodiversiteQuebec/gbif_insert#egg=gbif_insert

Environment variables

Configuration is done through environment variables. You can set them in your shell or in a .env file.

To use a dotenv file, create a .env file in the root directory of the project. This file will contain the environment variables required to run the script.

GBIF_USER=... # GBIF account username
GBIF_PASSWORD=... # GBIF account password
GBIF_EMAIL=... # GBIF account email
DB_HOST=localhost # Database host
DB_PORT=5432 # Database port
DB_NAME=atlas # Database name
DB_USER=postgres # Database user
DB_PASSWORD= # Database password, optional

Usage

Command line

Script scrupts/run.sh can be used to download and inject data from gbif into atlas database. It reads the environment variables from the .env file.

./scripts/run.sh

How it works

There are 3 parameters that must be sent to the Gbif object regarding the gbif account: username, password and email. These must be a valid gbif account parameters in order to trigger the download. Those variables can be set in the .env file or in the constructor of the Gbif object as described in the configuration section.

Please refer to the example below to try out an injection test.

Test code example

# Script pour lire les .csv et les envoyer dans la BD
#from ast import Str
import time
import tempfile
import os

# Méthode d'entrée du programme
# In: aucun param (fonction main)
# Out: fin du programme
def main():
    # Create temp workdir for test
    temp_dir = tempfile.mkdtemp(prefix='gbif_insert_test_')
    cur_dir = os.getcwd()
    os.chdir(temp_dir)

    from gbif_insert.gbif import Gbif
    import os
    from dotenv import load_dotenv

    # Load environment variables
    load_dotenv()
    GBIF_USER = os.getenv('GBIF_USER')
    GBIF_PASSWORD = os.getenv('GBIF_PASSWORD')
    GBIF_EMAIL = os.getenv('GBIF_EMAIL')

    # Calculer le temps d'execution
    start_time = time.time() 

    gbif_object = Gbif(GBIF_USER, GBIF_PASSWORD, GBIF_EMAIL)
    
    # Téléchargement des données, validation et injection
    gbif_object.start()

    print("Temps d'exécution: " + str(time.time() - start_time) + " secondes.")
    os.chdir(cur_dir)                                                                                                   

# Point d'entrée
main()