Plgdata

API Documentation

Note: all following operations require so-called user proxy certificate. More information related to that certificate is provided in the end of this documentation. All following code samples assume, that the proxy certificate resides in grid_proxy file in the current catalogue. That certificate may be passed as a parameter of a call named proxy or inside a header named PROXY. Both these options are shown in examples below. In the second case, however, the proxy certificate needs to be properly encoded.

All described remote operations on the intermediate PLGData server are performed using the HTTPS protocol, by calling a specified HTTP verb (e.g. GET or POST) at a given URL, accompanied by required parameters.

Paths to folders and files presented in examples below refer to resources stored on disks of the cluster identified by site_key. Currently following clusters are suppprted:

Ares - site_key: ares.

Listing a folder

GET:: https://data.plgrid.pl/list/[site_key]/[folder_path]

As a call parameter one should pass the content of the proxy certificate in parameter proxy. Calling this operation returns a JSON document that lists content of the specified folder, including subfolders and hidden files. The resulting document holds a list of elements, one per subfolder or file, with the following information:

rights: The list of access rights in the POSIX format , that is drwxrwxrwx with the - character denoting lack of certain rights
owner: User name of the owner of the file or folder. This user access rights are specified with the first triad in the rights field
group: The system user group that owns the file or folder. For every member of that group the access rights are presented by the second triad in the rights field
size: Size of the file in bytes
modification_date: The last modification date, given in English, in the following format: MMM [D]D HH:mm
name: Name of the file or folder
is_dir: A boolean flag that tells if it's a file or a folder. Similar information could be obtained by checking the first letter of the rights field

Sample element of a returned JSON document:

{"rights":"drwxr-xr-x","owner":"plguserlogin","group":"plgrid","size":2048,"modification_date":"May 6 18:24","name":"my_folder","is_dir":true}

Sample operation call using the curl command line tool:

curl -X GET https://data.plgrid.pl/list/[site_key]/net/people/plguserlogin/ --data-urlencode proxy="`cat grid_proxy`"

Same thing but with passing the encoded content of the proxy certificate in a header:

curl -X GET https://data.plgrid.pl/list/[site_key]/net/people/plguserlogin/ -H "PROXY:`cat grid_proxy | base64 | tr -d '\n'`"

Downloading a file

GET:: https://data.plgrid.pl/download/[site_key]/[file_path]

As a call parameter one should pass the content of the proxy certificate in parameter proxy. The operation starts downloading content of a file from the cluster, through a secure stream. The client is responsible to properly handle such stream on its side, and save the passed data. Sample operation call using the curl command line tool:

curl -X GET https://data.plgrid.pl/download/[site_key]/net/people/plguserlogin/graph.png
--data-urlencode proxy="`cat grid_proxy`"

Uploading a file

POST:: https://data.plgrid.pl/upload/[site_key]/[target_folder_path]

Currenty there is a limit of 256 MB of files uploaded to a cluster through API. The operation assumes the following parameters to the POST call:

file: The file to be uploaded, as in the case of POSTing a multipart form
proxy: Content of the proxy certificate of a user on behalf of whom the file is being uploaded
locale: An optional parameter that should be set to 'en' in order to get the response messages in English
recursive: An optional parameter which forces creation of the target folder, recursively (that is, including all intermediate folders), if the target folder does not exist.

Sample operation call using the curl command line tool:

curl -X POST https://data.plgrid.pl/upload/[site_key]/net/people/plguserlogin/zzzz/ -F proxy="`cat grid_proxy`"
-F "file=@graph.png"

Again, the same but using a header to pass the encoded content of the proxy certificate:

curl -X POST https://data.plgrid.pl/upload/[site_key]/net/people/plguserlogin/zzzz/
-H "PROXY:`cat grid_proxy | base64 | tr -d '\n'`" -F "file=@graph.png"

An example with the recursive alternative:

curl -X POST https://data.plgrid.pl/upload/[site_key]/net/people/plguserlogin/zzzz/1/2/3/ -F proxy="`cat grid_proxy`"
-F "file=@graph.png" -F locale=en -F recursive=true

New folder creation

POST:: https://data.plgrid.pl/mkdir/[site_key]/[target_folder_path]

This operation creates a new folder under given path in the cluster file system. There are following parameters:

proxy: Content of the proxy certificate of a user on behalf of whom the folder is being created
recursive: An optional parameter that forces creation of all intermediate folders, that are not existing, in a recursive manner (that is, all inexisting folders in the given target path will be created).

Sample operation call using the curl command line tool:

curl -X POST https://data.plgrid.pl/mkdir/[site_key]/net/people/plguserlogin/zzz/eeee -F proxy="`cat grid_proxy`"

curl -X POST https://data.plgrid.pl/mkdir/[site_key]/net/people/plguserlogin/zzz/eeee/1/2 -F proxy="`cat grid_proxy`"
-F recursive=true

Removing a file or a folder

DELETE:: https://data.plgrid.pl/remove/[site_key]/[file_or_folder_path]

The operation deletes a file or a folder from the cluster file system. There are following parameters:

proxy: Content of the proxy certificate of a user on behalf of whom the file or folder is being deleted
is_dir: A required parameter that should be set to true, if we want to remove a folder. This extra information is required as the GridFTP protocol, which is used by PLGData to communicate with the cluster file system, distinguishes the file removal and the folder removal operations.
locale: An optional parameter that should be set to 'en' in order to get the response messages in English

Sample operation call using the curl command line tool:

curl -X DELETE https://data.plgrid.pl/remove/[site_key]/net/people/plguserlogin/graph.png -F proxy="`cat grid_proxy`"

curl -X DELETE https://data.plgrid.pl/remove/[site_key]/net/people/plguserlogin/zzz/ -F proxy="`cat grid_proxy`"
-F "is_dir=true"

Transferring (copying) a file

PUT:: https://data.plgrid.pl/transfer/[site_key]/[source_file_path]

The operation copies a file from the cluster file system to different location. There are following parameters:

proxy: Content of the proxy certificate of a user on behalf of whom the file is being copied
destination_path: A required parameter that should be point to a place where the copy of the file needs to be saved. It also needs the name of the file.
locale: An optional parameter that should be set to 'en' in order to get the response messages in English

Sample operation call using the curl command line tool:

curl -X PUT https://data.plgrid.pl/transfer/[site_key]/net/people/plguserlogin/dir1/file.txt -F proxy="`cat grid_proxy`"
-F "destination_path=/net/people/plguserlogin/dir2/copied_file.txt"

PL-Grid user proxy certificate

The X509 user proxy certificate is generated each time a user logs in the PLGData service (the second password field). In order to use the presented API, such a proxy certificate should be stored locally, for instance in a file (or in a memory), on the computer where the code that integrates with PLGData API runs.

The simplest way to generate such a certificate is to issue the following command:

read -s p && echo $p | ssh -l plguserlogin pro.cyfronet.pl "grid-proxy-init -q -pwstdin && cat /tmp/x509up_u\`id -u\`" > grid_proxy && unset p

Don't forget to change plguserlogin to your PLGrid login. First of all you'll need to type in your passphrase to your private key (the second password from the OpenId PLGData login procedure). There will be no prompt for this, but your input will be masked (not shown in the screen). Then, you will be asked for the regular login to your PLGrid account, i.e. you will need to supply your plguserlogin account password. After the command was executed, the generated proxy certificate should be stored in the current directory, in the grid_proxy file. It should be valid for the next 12 hours.