REST API Examples

REST API Examples#

In this notebook we explore searching for metadata from the REST API. The REST API provides a method to programmatically extract a JSON representation of the meta data from the API.

First we load some python dependancies that we will use as part of this notebook and set the variable API_URL to the location of the REST API.

import pandas as pd
import requests
API_URL = "https://mastapp.site/json"

Below we print the location of the endpoint we will query in this notebook:

print(f"REST API Endpoint: {API_URL}")

REST API Endpoint: https://mastapp.site/json

Querying Shots with the REST API#

We’re going to use the python requests library to query metadata from the database. All we need to do to get a result is to query the database with a HTTP GET at the appropriate endpoint. For example, to get information about different experimental shots we can query the /json/shots/ endpoint.

response = requests.get(f'{API_URL}/shots')
result = response.json()
print(f"Query returned status code: {response.status_code}")

Query returned status code: 200

The shots endpoint returns a JSON payload with a list of shots. Let’s look at the first element from the payload:

result['items'][0]

{'shot_id': 11695,
 'uuid': '219e5d59-b9dc-584f-b83b-018c47e8739d',
 'url': 's3://mast/level1/shots/11695.zarr',
 'timestamp': '2004-12-13T11:54:00',
 'preshot_description': '\n0.1T TF SHOT\n',
 'postshot_description': '\nOK\n',
 'campaign': 'M5',
 'reference_shot': None,
 'scenario': None,
 'heating': None,
 'pellets': False,
 'rmp_coil': None,
 'current_range': None,
 'divertor_config': 'Conventional',
 'plasma_shape': None,
 'comissioner': None,
 'facility': 'MAST',
 'cpf_p03249': None,
 'cpf_p04673': None,
 'cpf_p04674': None,
 'cpf_p04675': None,
 'cpf_p04676': None,
 'cpf_p04677': None,
 'cpf_p04678': None,
 'cpf_p04679': None,
 'cpf_p04680': None,
 'cpf_p04681': None,
 'cpf_p04833': None,
 'cpf_p04993': None,
 'cpf_p05007': None,
 'cpf_p05008': None,
 'cpf_p05009': None,
 'cpf_p05010': None,
 'cpf_p05011': None,
 'cpf_p05015': None,
 'cpf_p05016': None,
 'cpf_p05017': None,
 'cpf_p05025': None,
 'cpf_p05027': None,
 'cpf_p05028': None,
 'cpf_p05029': None,
 'cpf_p05030': None,
 'cpf_p05032': None,
 'cpf_p05033': None,
 'cpf_p05153': None,
 'cpf_p06000': None,
 'cpf_p06001': None,
 'cpf_p06002': None,
 'cpf_p06003': None,
 'cpf_p06004': None,
 'cpf_p10963': None,
 'cpf_p10964': None,
 'cpf_p12441': None,
 'cpf_p12450': None,
 'cpf_p12451': None,
 'cpf_p12452': None,
 'cpf_p15202': None,
 'cpf_p15203': None,
 'cpf_p15209': None,
 'cpf_p15659': None,
 'cpf_p15660': None,
 'cpf_p15661': None,
 'cpf_p20000': None,
 'cpf_p20204': None,
 'cpf_p20205': None,
 'cpf_p20206': None,
 'cpf_p20207': None,
 'cpf_p20208': None,
 'cpf_p21010': None,
 'cpf_p21011': None,
 'cpf_p21012': None,
 'cpf_p21021': None,
 'cpf_p21022': None,
 'cpf_p21029': None,
 'cpf_p21035': None,
 'cpf_p21037': None,
 'cpf_p21041': None,
 'cpf_p21042': None,
 'cpf_p21043': None,
 'cpf_p21044': None,
 'cpf_p21045': None,
 'cpf_p21046': None,
 'cpf_p21047': None,
 'cpf_p21048': None,
 'cpf_p21051': None,
 'cpf_p21052': None,
 'cpf_p21053': None,
 'cpf_p21054': None,
 'cpf_p21055': None,
 'cpf_p21056': None,
 'cpf_p21075': None,
 'cpf_p21076': None,
 'cpf_p21077': None,
 'cpf_p21078': None,
 'cpf_p21079': None,
 'cpf_p21080': None,
 'cpf_p21081': None,
 'cpf_p21082': None,
 'cpf_p21083': None,
 'cpf_p21084': None,
 'cpf_p21085': None,
 'cpf_p21086': None,
 'cpf_p21087': None,
 'cpf_p21088': None,
 'cpf_p21089': None,
 'cpf_p21092': None,
 'cpf_p21093': None,
 'cpf_abort': None,
 'cpf_amin_ipmax': None,
 'cpf_amin_max': None,
 'cpf_amin_truby': None,
 'cpf_area_ipmax': None,
 'cpf_area_max': None,
 'cpf_area_truby': None,
 'cpf_bepmhd_ipmax': None,
 'cpf_bepmhd_max': None,
 'cpf_bepmhd_truby': None,
 'cpf_betmhd_ipmax': None,
 'cpf_betmhd_max': None,
 'cpf_betmhd_truby': None,
 'cpf_bt_ipmax': None,
 'cpf_bt_max': None,
 'cpf_bt_truby': None,
 'cpf_c2ratio': None,
 'cpf_column_temp_in': None,
 'cpf_column_temp_out': None,
 'cpf_creation': None,
 'cpf_dwmhd_ipmax': None,
 'cpf_dwmhd_max': None,
 'cpf_dwmhd_truby': None,
 'cpf_enbi_max_ss': None,
 'cpf_enbi_max_sw': None,
 'cpf_exp_date': None,
 'cpf_exp_number': None,
 'cpf_exp_time': None,
 'cpf_gdc_duration': None,
 'cpf_gdc_time': None,
 'cpf_ibgas_pressure': None,
 'cpf_ip_av': None,
 'cpf_ip_max': None,
 'cpf_jnbi_ipmax': None,
 'cpf_jnbi_ipmax_ss': None,
 'cpf_jnbi_ipmax_sw': None,
 'cpf_jnbi_max': None,
 'cpf_jnbi_max_ss': None,
 'cpf_jnbi_max_sw': None,
 'cpf_jnbi_total': None,
 'cpf_jnbi_total_ss': None,
 'cpf_jnbi_total_sw': None,
 'cpf_jnbi_truby': None,
 'cpf_jnbi_truby_ss': None,
 'cpf_jnbi_truby_sw': None,
 'cpf_johm_ipmax': None,
 'cpf_johm_max': None,
 'cpf_johm_total': None,
 'cpf_johm_truby': None,
 'cpf_kappa_ipmax': None,
 'cpf_kappa_max': None,
 'cpf_kappa_truby': None,
 'cpf_li_2_ipmax': None,
 'cpf_li_2_max': None,
 'cpf_li_2_truby': None,
 'cpf_li_3_ipmax': None,
 'cpf_li_3_max': None,
 'cpf_li_3_truby': None,
 'cpf_log_base_pressure': None,
 'cpf_ndl_co2_ipmax': None,
 'cpf_ndl_co2_max': None,
 'cpf_ndl_co2_truby': None,
 'cpf_ne0_ipmax': None,
 'cpf_ne0_max': None,
 'cpf_ne0_truby': None,
 'cpf_ne0ratio_ipmax': None,
 'cpf_ne0ruby': None,
 'cpf_ne_bar_ipmax': None,
 'cpf_ne_yag_bar_ipmax': None,
 'cpf_ngreenwald_ipmax': None,
 'cpf_ngreenwaldratio_ipmax': None,
 'cpf_o2ratio': None,
 'cpf_objective': None,
 'cpf_pe0_ipmax': None,
 'cpf_pe0_max': None,
 'cpf_pe0_truby': None,
 'cpf_pe0ruby': None,
 'cpf_pic': None,
 'cpf_pnbi_ipmax': None,
 'cpf_pnbi_ipmax_ss': None,
 'cpf_pnbi_ipmax_sw': None,
 'cpf_pnbi_max': None,
 'cpf_pnbi_max_ss': None,
 'cpf_pnbi_max_sw': None,
 'cpf_pnbi_truby': None,
 'cpf_pnbi_truby_ss': None,
 'cpf_pnbi_truby_sw': None,
 'cpf_pohm_ipmax': None,
 'cpf_pohm_max': None,
 'cpf_pohm_truby': None,
 'cpf_postshot': None,
 'cpf_prad_ipmax': None,
 'cpf_prad_max': None,
 'cpf_prad_truby': None,
 'cpf_pradne2': None,
 'cpf_preshot': None,
 'cpf_program': None,
 'cpf_pulno': None,
 'cpf_q95_ipmax': None,
 'cpf_q95_min': None,
 'cpf_q95_truby': None,
 'cpf_reference': None,
 'cpf_rgeo_ipmax': None,
 'cpf_rgeo_max': None,
 'cpf_rgeo_truby': None,
 'cpf_rinner_da': None,
 'cpf_rinner_efit': None,
 'cpf_rmag_efit': None,
 'cpf_router_da': None,
 'cpf_router_efit': None,
 'cpf_sarea_ipmax': None,
 'cpf_sarea_max': None,
 'cpf_sarea_truby': None,
 'cpf_sl': None,
 'cpf_summary': None,
 'cpf_tamin_max': None,
 'cpf_tarea_max': None,
 'cpf_tautot_ipmax': None,
 'cpf_tautot_max': None,
 'cpf_tautot_truby': None,
 'cpf_tbepmhd_max': None,
 'cpf_tbetmhd_max': None,
 'cpf_tbt_max': None,
 'cpf_tdwmhd_max': None,
 'cpf_te0_ipmax': None,
 'cpf_te0_max': None,
 'cpf_te0_truby': None,
 'cpf_te0ratio_ipmax': None,
 'cpf_te0ruby': None,
 'cpf_te_yag_bar_ipmax': None,
 'cpf_tend': None,
 'cpf_tend_ibgas': None,
 'cpf_tend_nbi': None,
 'cpf_tend_nbi_ss': None,
 'cpf_tend_nbi_sw': None,
 'cpf_term_code': None,
 'cpf_tftend': None,
 'cpf_tftstart': None,
 'cpf_tipmax': None,
 'cpf_tkappa_max': None,
 'cpf_tli_2_max': None,
 'cpf_tli_3_max': None,
 'cpf_tndl_co2_max': None,
 'cpf_tne0_max': None,
 'cpf_tpe0_max': None,
 'cpf_tpnbi_max': None,
 'cpf_tpnbi_max_ss': None,
 'cpf_tpnbi_max_sw': None,
 'cpf_tpohm_max': None,
 'cpf_tprad_max': None,
 'cpf_tq95_min': None,
 'cpf_trgeo_max': None,
 'cpf_truby': None,
 'cpf_tsarea_max': None,
 'cpf_tstart': None,
 'cpf_tstart_ibgas': None,
 'cpf_tstart_nbi': None,
 'cpf_tstart_nbi_ss': None,
 'cpf_tstart_nbi_sw': None,
 'cpf_ttautot_max': None,
 'cpf_tte0_max': None,
 'cpf_tvol_max': None,
 'cpf_twmhd_max': None,
 'cpf_tzeff_max': None,
 'cpf_useful': None,
 'cpf_vol_ipmax': None,
 'cpf_vol_max': None,
 'cpf_vol_truby': None,
 'cpf_wmhd_ipmax': None,
 'cpf_wmhd_max': None,
 'cpf_wmhd_truby': None,
 'cpf_zeff_ipmax': None,
 'cpf_zeff_max': None,
 'cpf_zeff_truby': None,
 'cpf_zmag_efit': None}

Each item in the list is a json object. This contains the meta-data items that corresponded to our query. In this case, each item contains information about a different MAST shot. Each item has lots of information about different shots, for example the shot ID, the campaign the shot was part of, the pre- and post-shot description by investigators.

For more information on the what’s returned by the API you can look at the endpoint documentation:

https://mast-app.site/redoc

Of course, we can read all this JSON data directly into common python data analysis packages, for example, we can create a pandas dataframe directly from the endpoint data

df = pd.read_json(f'{API_URL}/shots')['items']
df = pd.DataFrame(df.to_list())
df.head()

	shot_id	uuid	url	timestamp	preshot_description	postshot_description	campaign	reference_shot	scenario	heating	...	cpf_vol_ipmax	cpf_vol_max	cpf_vol_truby	cpf_wmhd_ipmax	cpf_wmhd_max	cpf_wmhd_truby	cpf_zeff_ipmax	cpf_zeff_max	cpf_zeff_truby	cpf_zmag_efit
0	11695	219e5d59-b9dc-584f-b83b-018c47e8739d	s3://mast/level1/shots/11695.zarr	2004-12-13T11:54:00	\n0.1T TF SHOT\n	\nOK\n	M5	None	None	None	...	None	None	None	None	None	None	None	None	None	None
1	11696	9ba69c96-34ac-5cd4-a1b8-8e9d7a44f871	s3://mast/level1/shots/11696.zarr	2004-12-13T12:07:00	\nSTANDARD 0.3T TF SHOT\n	\nOK\n	M5	None	None	None	...	None	None	None	None	None	None	None	None	None	None
2	11697	6ecd5c59-6d16-5a0f-a715-be31a88ca34e	s3://mast/level1/shots/11697.zarr	2004-12-13T12:19:00	\nRAISE TO 0.5T\n	\nOK, ALARMS ARE LOWER\n	M5	None	None	None	...	None	None	None	None	None	None	None	None	None	None
3	11698	94366fda-b75e-5046-bb44-401844c563d8	s3://mast/level1/shots/11698.zarr	2004-12-13T12:31:00	\nRAISE TO .56T\n	\nSTILL ALARMS BUT LOWER AGAIN\n	M5	None	None	None	...	None	None	None	None	None	None	None	None	None	None
4	11699	0e0f522d-a17f-5575-9bd0-d931231e8a71	s3://mast/level1/shots/11699.zarr	2004-12-13T12:45:00	\nRAISE TO .58T\n	\nOK\n	M5	None	None	None	...	None	None	None	None	None	None	None	None	None	None

5 rows × 282 columns

Searching & Filtering Data#

All REST API endpoints can take query parameters to filter the data returned. For example, we can return all shots for the M9 campaign by using the approrpiate query string.

For example, we can query for everything from the M9 campaign by adding ?campaign=M9 to our query string.

df = pd.read_json(f"{API_URL}/shots?campaign=M9")['items']
df = pd.DataFrame(df.to_list())
df.head()

	shot_id	uuid	url	timestamp	preshot_description	postshot_description	campaign	reference_shot	scenario	heating	...	cpf_vol_ipmax	cpf_vol_max	cpf_vol_truby	cpf_wmhd_ipmax	cpf_wmhd_max	cpf_wmhd_truby	cpf_zeff_ipmax	cpf_zeff_max	cpf_zeff_truby	cpf_zmag_efit
0	11695	219e5d59-b9dc-584f-b83b-018c47e8739d	s3://mast/level1/shots/11695.zarr	2004-12-13T11:54:00	\n0.1T TF SHOT\n	\nOK\n	M5	None	None	None	...	None	None	None	None	None	None	None	None	None	None
1	11696	9ba69c96-34ac-5cd4-a1b8-8e9d7a44f871	s3://mast/level1/shots/11696.zarr	2004-12-13T12:07:00	\nSTANDARD 0.3T TF SHOT\n	\nOK\n	M5	None	None	None	...	None	None	None	None	None	None	None	None	None	None
2	11697	6ecd5c59-6d16-5a0f-a715-be31a88ca34e	s3://mast/level1/shots/11697.zarr	2004-12-13T12:19:00	\nRAISE TO 0.5T\n	\nOK, ALARMS ARE LOWER\n	M5	None	None	None	...	None	None	None	None	None	None	None	None	None	None
3	11698	94366fda-b75e-5046-bb44-401844c563d8	s3://mast/level1/shots/11698.zarr	2004-12-13T12:31:00	\nRAISE TO .56T\n	\nSTILL ALARMS BUT LOWER AGAIN\n	M5	None	None	None	...	None	None	None	None	None	None	None	None	None	None
4	11699	0e0f522d-a17f-5575-9bd0-d931231e8a71	s3://mast/level1/shots/11699.zarr	2004-12-13T12:45:00	\nRAISE TO .58T\n	\nOK\n	M5	None	None	None	...	None	None	None	None	None	None	None	None	None	None

5 rows × 282 columns

Pagination#

The REST API responses are paginated, meaning that only a subset of the full items are returned with each query. Pagination is used to limit the total number of requests made by each user to prevent any single user overloading the server with huge data requests.

Pagination information is included in the response and corresponds to RFC 8288. The response contains cursors at the bottom of the response:

current_page - this is the cursor relating to the current page of results.
next_page - this is the cursor for the next page of results.
previous_page - this is the cursor for the previous page of results, will result in null if on the first page.

You can control the pagination using the following options as query arguments:

cursor=xx to set the cursor for the displayed set of results
size=xx to set the number of results returned by each page.

response = requests.get(f'{API_URL}/shots?size=2&cursor=Pmk6MzAxMTE%3D')
result = response.json()
headers = response.headers

print("Current page cursor", result['current_page'])
print("Next page cursor", result['next_page'])
print("Previous page cursor", result['previous_page'])

df = pd.DataFrame(result['items'])
df

Current page cursor Pmk6MzAxMTE%3D
Next page cursor Pmk6MzAxMTM%3D
Previous page cursor PGk6MzAxMTI%3D

	shot_id	uuid	url	timestamp	preshot_description	postshot_description	campaign	reference_shot	scenario	heating	...	cpf_vol_ipmax	cpf_vol_max	cpf_vol_truby	cpf_wmhd_ipmax	cpf_wmhd_max	cpf_wmhd_truby	cpf_zeff_ipmax	cpf_zeff_max	cpf_zeff_truby	cpf_zmag_efit
0	30112	eefe2467-69b0-5fe7-9863-e21289daa135	s3://mast/level1/shots/30112.zarr	2013-09-09T11:42:00	\nReload 30110 - reduce IP flat top to 750 KA ...	\nPIC happy with data.\n	M9	30110	4	SS Beam	...	7.845118	8.841636	0.0	65898.305250	82056.664	0.0	None	None	None	0.047497
1	30113	742c78fa-3811-5404-93b3-5ab419d330bd	s3://mast/level1/shots/30113.zarr	2013-09-09T12:00:00	\nRestore 30111\n	\nShot OK for programme. Two good beams. Big l...	M9	30111	3	2 Beams,SS Beam,SW Beam	...	8.391672	8.426641	0.0	43127.703046	60260.926	0.0	None	None	None	0.042220

2 rows × 282 columns

REST API Examples

Contents

REST API Examples#

Querying Shots with the REST API#

Searching & Filtering Data#

Pagination#