Procedures and examples on downloading using registered downloads (as opposite to the download without registration through e.g. rgbif). This has several advantages. The most prominent are: 

* You are not restricted to a hard limit of 200 000 records
* You get a citation of your downloaded data that can/must be used when citing your data usage in e.g. a scientific publication or thesis. 

Due to the latter, **you should always use asyncronous downloads when fetching data for use in publications**, at least when your analyses are based upon data from parts of multiple datasets. Your specific download-request will be assigned a DOI which will resolve to the exact same representation of the data as used creating a citable, reproducible workflow for your analyses. GBIF will keep the data for this DOI for a prolonged time, "forever" if this DOI appears in a publication. By using the assigned DOIs included with your citations, you also vastly improve GBIF’s ability to [track the use of data](https://www.gbif.org/literature-tracking). It also provides the mechanism for connecting published uses of the data back to each source record of data. In addition to acknowledging data publishers, the practice of using DOI citations rewards them by reinforcing the value of sharing open data to the publisher’s stakeholders and .

In the following we go through the procedure of asynchronous download using the rgbif package step by step: 

1. load required packages
2. Register your GBIF user information (username, email and password) to your R session
3. Perform a simple search for getting example data, including find species of interest, construct query and finally request a download key from GBIF
4) Download data
5) Unzip and create a data.frame 

## Load packages

```{r loadpackages, eval=TRUE,message=FALSE,warning=FALSE}
library(rgbif)
library(stringr) # string manipulations (not needed, may also be done by base R)
library(rio) # data import (not needed, may also be done by base R)
library(dplyr) # for data-wrangling
library(wicket)

```

## set user_name, e-mail, and pswd as global options first
It is necessary to register as a GIBF user to create a download request (see https://www.gbif.org/developer/occurrence#download). You can create your user for free at the [GBIF website](https://www.gbif.org)

You can do this directly in the occ_download call below, however, here we for convenience put it in global options of your R session so that you don't have to hard-code passwords into scripts (always a bad idea). You only need to do this once per R session. The below chunk requires that you run your script through R-studio. 

```{r registeruser, eval=FALSE}
options(gbif_user=rstudioapi::askForPassword("my gbif username"))
options(gbif_email=rstudioapi::askForPassword("my registred gbif e-mail"))
options(gbif_pwd=rstudioapi::askForPassword("my gbif password"))
```


## Search and request download key
Here we request a download key by sending an API call to GBIF using the occ_download function in rgbif. The download key will later be used to download the data

first find a species key using the name_suggest function and create a polygon for spatial filtering (for convenience, we do this step outside the request for download key). Secondly, we create the query and request the download key.

Note that there may be some time-delay before GBIF are able to handle your request depending on the size of your data. Remember that GBIF will only handle a maximum of 3 download request per user simultaneously. You will receive a e-mail with confirmation when your download request is processed or find an overview of all your requested and finalized downloads at https://www.gbif.org/user/download. 

```{r searchandgetkey, eval=FALSE}

# Find a taxonkey - get list of gbif keys to filter download
key <- name_suggest(q='Esox lucius', rank='species')$key[1] 

# Crate spatial filter
my_wkt <- "POLYGON((10.32989501953125 63.26787016946243, 10.32989501953125 63.455051146616825, 10.8819580078125 63.455051146616825, 10.8819580078125 63.26787016946243, 10.32989501953125 63.26787016946243))" 
#wicket::validate_wkt(my_wkt)
geom_param <- paste("geometry", "within", my_wkt)


# Get download key. NB! Maximum of 3 download requests handled simultaneously
download_key <- 
  occ_download(
    'taxonKey = 2346633',
    'hasCoordinate = TRUE',
    geom_param,
    type = "and"
  ) %>% 
  occ_download_meta


```

## Download data 
Two ways to do this, you can either wait for you confirmation e-mail or just try by simply pasting the download key into an URL string. If you want to rather just run your scrips and have a cup of coffee before your workflow finalizes, or want to set up a workflow that runs automatically e.g. as a nightly cron-job you may need something slightly more sophisticated. Below we therefore give a short but inelegant demo for how to put the download into a time-delay function. 

The current script is set up to store the downloaded data to a temporary file. If you work with large datasets you probably want to store the data on disk for re-use in later sessions.  

```{r download, eval=FALSE}
# a) Direct download by link: The download key will be shown as last part of the url e.g. https://www.gbif.org/occurrence/download/0003580-171002173027117

tmp <- tempfile() # create temporary file for download
download.file(url=paste("http://api.gbif.org/v1/occurrence/download/request/",
                        download_key[1],sep=""),
              destfile=tmp,
              quiet=FALSE)

# b) The coffebreak version
#------------------------------------------------------------------------------
# Asyncronous download from the GBIF API. 
# The function tries, with given time intervall to download data generated 
# by the download key as objec given by "occ_download" function of the "rgbif" 
# library. 
#
# Input:
# download_key: GBIF API download key (e.g. from occ_download in rgbif package)
# n_try: Number of times the function should keep trying
# Sys.sleep_duration: Time interval between each try
#
# Output: 
# Downloaded dwc-archive, named with the download key and written to the 
# the current R session working directory.
#------------------------------------------------------------------------------

download_GBIF_API <- function(download_key,n_try,Sys.sleep_duration,destfile_name){
  start_time <- Sys.time()
  n_try_count <- 1
  
  download_url <- paste("http://api.gbif.org/v1/occurrence/download/request/",
                        download_key[1],sep="")
  
  try_download <- try(download.file(url=download_url,destfile=destfile_name,
                                    quiet=TRUE),silent = TRUE)
  
  while (inherits(try_download, "try-error") & n_try_count < n_try) {   
    Sys.sleep(Sys.sleep_duration)
    n_try_count <- n_try_count+1
    try_download <- try(download.file(url=download_url,destfile=destfile_name,
                      quiet=TRUE),silent = TRUE)
    print(paste("trying... Download link not ready. Time elapsed (min):",
                round(as.numeric(paste(difftime(Sys.time(),start_time, units = "mins"))),2)))
  }
}


# call function
tmp <- tempfile() # create temporary file for download
download_GBIF_API(download_key=download_key,destfile_name=tmp,n_try=5,Sys.sleep_duration=30)

```


## Open the data and extract into data.frame 
The download gives us back a package with data and metadata bundled together in a .zip file. This includes both the metadata, citations of the original datasets that the occurrence download is a composite of, the licenses, as well as the data in both gbif interpreted form (occurrence.txt) and the raw data as provided by the user (verbatim.txt). It is usually the interpreted data you want to use (found in the file occurrence.txt). 

```{r getdata, eval=FALSE}
# Get a list of the files within the archive by using "list=TRUE" in the unzip function.
archive_files <- unzip(tmp, files = "NULL", list = T) 

# Get the occurrence.txt file in as a dataframe (using import from rio)
occurrence <- import(unzip(tmp,
                 files="occurrence.txt"),header=T,sep="\t")
```

## Cite your data! 
Finally but not at least, remember to cite your data properly:

```{r citation, eval=FALSE}
paste("GBIF Occurrence Download", download_key[2], "accessed via GBIF.org on", Sys.Date())
```
