{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "76fa365c-b6d9-479f-99d8-d0d53ec30e8e",
   "metadata": {},
   "source": [
    "# REST API Examples\n",
    "\n",
    "In this notebook we explore searching for metadata from the REST API. The REST API provides a method to programmatically extract a JSON representation of the meta data from the API.\n",
    "\n",
    "First we load some python dependancies that we will use as part of this notebook and set the variable `API_URL` to the location of the REST API."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "ebba03b7-34c4-4b49-94d8-112c7176ac7f",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import requests\n",
    "API_URL = \"https://mastapp.site/json\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3408c42d",
   "metadata": {},
   "source": [
    "Below we print the location of the endpoint we will query in this notebook:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ecbd2ced",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "REST API Endpoint: https://mastapp.site/json\n"
     ]
    }
   ],
   "source": [
    "print(f\"REST API Endpoint: {API_URL}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba8f861b-b74f-40d1-9df2-6b4f5b8f6edd",
   "metadata": {},
   "source": [
    "## Querying Shots with the REST API\n",
    "\n",
    "We're going to use the python `requests` library to query metadata from the database. All we need to do to get a result is to query the database with a HTTP GET at the appropriate endpoint. For example, to get information about different experimental shots we can query the `/json/shots/` endpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "a722af66-9731-4878-b2df-9c9ba7ac0be7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Query returned status code: 200\n"
     ]
    }
   ],
   "source": [
    "response = requests.get(f'{API_URL}/shots')\n",
    "result = response.json()\n",
    "print(f\"Query returned status code: {response.status_code}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "473e3069",
   "metadata": {},
   "source": [
    "The shots endpoint returns a JSON payload with a list of shots. Let's look at the first element from the payload:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "2611f771",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'shot_id': 15085,\n",
       " 'uuid': '0bb8ea1b-954b-5cce-8b2d-df40a0703b2f',\n",
       " 'url': 's3://mast/level1/shots/15085.zarr',\n",
       " 'timestamp': '2006-04-25T13:48:00',\n",
       " 'preshot_description': '\\nReplace ccbv connector. repeat.\\n',\n",
       " 'postshot_description': '\\nRadial control unchanged.  Zip/FA jump less pronounced and earlier. 1st sawtooth new nearly as late as 15008.\\n',\n",
       " 'campaign': 'M6',\n",
       " 'reference_shot': 15084,\n",
       " 'scenario': None,\n",
       " 'heating': None,\n",
       " 'pellets': False,\n",
       " 'rmp_coil': None,\n",
       " 'current_range': None,\n",
       " 'divertor_config': 'Conventional',\n",
       " 'plasma_shape': None,\n",
       " 'comissioner': None,\n",
       " 'facility': 'MAST',\n",
       " 'cpf_p03249': None,\n",
       " 'cpf_p04673': None,\n",
       " 'cpf_p04674': None,\n",
       " 'cpf_p04675': None,\n",
       " 'cpf_p04676': None,\n",
       " 'cpf_p04677': None,\n",
       " 'cpf_p04678': None,\n",
       " 'cpf_p04679': None,\n",
       " 'cpf_p04680': None,\n",
       " 'cpf_p04681': None,\n",
       " 'cpf_p04833': None,\n",
       " 'cpf_p04993': None,\n",
       " 'cpf_p05007': None,\n",
       " 'cpf_p05008': None,\n",
       " 'cpf_p05009': None,\n",
       " 'cpf_p05010': None,\n",
       " 'cpf_p05011': None,\n",
       " 'cpf_p05015': None,\n",
       " 'cpf_p05016': None,\n",
       " 'cpf_p05017': None,\n",
       " 'cpf_p05025': None,\n",
       " 'cpf_p05027': None,\n",
       " 'cpf_p05028': None,\n",
       " 'cpf_p05029': None,\n",
       " 'cpf_p05030': None,\n",
       " 'cpf_p05032': None,\n",
       " 'cpf_p05033': None,\n",
       " 'cpf_p05153': None,\n",
       " 'cpf_p06000': None,\n",
       " 'cpf_p06001': None,\n",
       " 'cpf_p06002': None,\n",
       " 'cpf_p06003': None,\n",
       " 'cpf_p06004': None,\n",
       " 'cpf_p10963': None,\n",
       " 'cpf_p10964': None,\n",
       " 'cpf_p12441': None,\n",
       " 'cpf_p12450': None,\n",
       " 'cpf_p12451': None,\n",
       " 'cpf_p12452': None,\n",
       " 'cpf_p15202': None,\n",
       " 'cpf_p15203': None,\n",
       " 'cpf_p15209': None,\n",
       " 'cpf_p15659': None,\n",
       " 'cpf_p15660': None,\n",
       " 'cpf_p15661': None,\n",
       " 'cpf_p20000': None,\n",
       " 'cpf_p20204': None,\n",
       " 'cpf_p20205': None,\n",
       " 'cpf_p20206': None,\n",
       " 'cpf_p20207': None,\n",
       " 'cpf_p20208': None,\n",
       " 'cpf_p21010': None,\n",
       " 'cpf_p21011': None,\n",
       " 'cpf_p21012': None,\n",
       " 'cpf_p21021': None,\n",
       " 'cpf_p21022': None,\n",
       " 'cpf_p21029': None,\n",
       " 'cpf_p21035': None,\n",
       " 'cpf_p21037': None,\n",
       " 'cpf_p21041': None,\n",
       " 'cpf_p21042': None,\n",
       " 'cpf_p21043': None,\n",
       " 'cpf_p21044': None,\n",
       " 'cpf_p21045': None,\n",
       " 'cpf_p21046': None,\n",
       " 'cpf_p21047': None,\n",
       " 'cpf_p21048': None,\n",
       " 'cpf_p21051': None,\n",
       " 'cpf_p21052': None,\n",
       " 'cpf_p21053': None,\n",
       " 'cpf_p21054': None,\n",
       " 'cpf_p21055': None,\n",
       " 'cpf_p21056': None,\n",
       " 'cpf_p21075': None,\n",
       " 'cpf_p21076': None,\n",
       " 'cpf_p21077': None,\n",
       " 'cpf_p21078': None,\n",
       " 'cpf_p21079': None,\n",
       " 'cpf_p21080': None,\n",
       " 'cpf_p21081': None,\n",
       " 'cpf_p21082': None,\n",
       " 'cpf_p21083': None,\n",
       " 'cpf_p21084': None,\n",
       " 'cpf_p21085': None,\n",
       " 'cpf_p21086': None,\n",
       " 'cpf_p21087': None,\n",
       " 'cpf_p21088': None,\n",
       " 'cpf_p21089': None,\n",
       " 'cpf_p21092': None,\n",
       " 'cpf_p21093': None,\n",
       " 'cpf_abort': None,\n",
       " 'cpf_amin_ipmax': None,\n",
       " 'cpf_amin_max': None,\n",
       " 'cpf_amin_truby': None,\n",
       " 'cpf_area_ipmax': None,\n",
       " 'cpf_area_max': None,\n",
       " 'cpf_area_truby': None,\n",
       " 'cpf_bepmhd_ipmax': None,\n",
       " 'cpf_bepmhd_max': None,\n",
       " 'cpf_bepmhd_truby': None,\n",
       " 'cpf_betmhd_ipmax': None,\n",
       " 'cpf_betmhd_max': None,\n",
       " 'cpf_betmhd_truby': None,\n",
       " 'cpf_bt_ipmax': None,\n",
       " 'cpf_bt_max': None,\n",
       " 'cpf_bt_truby': None,\n",
       " 'cpf_c2ratio': None,\n",
       " 'cpf_column_temp_in': None,\n",
       " 'cpf_column_temp_out': None,\n",
       " 'cpf_creation': None,\n",
       " 'cpf_dwmhd_ipmax': None,\n",
       " 'cpf_dwmhd_max': None,\n",
       " 'cpf_dwmhd_truby': None,\n",
       " 'cpf_enbi_max_ss': None,\n",
       " 'cpf_enbi_max_sw': None,\n",
       " 'cpf_exp_date': None,\n",
       " 'cpf_exp_number': None,\n",
       " 'cpf_exp_time': None,\n",
       " 'cpf_gdc_duration': None,\n",
       " 'cpf_gdc_time': None,\n",
       " 'cpf_ibgas_pressure': None,\n",
       " 'cpf_ip_av': None,\n",
       " 'cpf_ip_max': None,\n",
       " 'cpf_jnbi_ipmax': None,\n",
       " 'cpf_jnbi_ipmax_ss': None,\n",
       " 'cpf_jnbi_ipmax_sw': None,\n",
       " 'cpf_jnbi_max': None,\n",
       " 'cpf_jnbi_max_ss': None,\n",
       " 'cpf_jnbi_max_sw': None,\n",
       " 'cpf_jnbi_total': None,\n",
       " 'cpf_jnbi_total_ss': None,\n",
       " 'cpf_jnbi_total_sw': None,\n",
       " 'cpf_jnbi_truby': None,\n",
       " 'cpf_jnbi_truby_ss': None,\n",
       " 'cpf_jnbi_truby_sw': None,\n",
       " 'cpf_johm_ipmax': None,\n",
       " 'cpf_johm_max': None,\n",
       " 'cpf_johm_total': None,\n",
       " 'cpf_johm_truby': None,\n",
       " 'cpf_kappa_ipmax': None,\n",
       " 'cpf_kappa_max': None,\n",
       " 'cpf_kappa_truby': None,\n",
       " 'cpf_li_2_ipmax': None,\n",
       " 'cpf_li_2_max': None,\n",
       " 'cpf_li_2_truby': None,\n",
       " 'cpf_li_3_ipmax': None,\n",
       " 'cpf_li_3_max': None,\n",
       " 'cpf_li_3_truby': None,\n",
       " 'cpf_log_base_pressure': None,\n",
       " 'cpf_ndl_co2_ipmax': None,\n",
       " 'cpf_ndl_co2_max': None,\n",
       " 'cpf_ndl_co2_truby': None,\n",
       " 'cpf_ne0_ipmax': None,\n",
       " 'cpf_ne0_max': None,\n",
       " 'cpf_ne0_truby': None,\n",
       " 'cpf_ne0ratio_ipmax': None,\n",
       " 'cpf_ne0ruby': None,\n",
       " 'cpf_ne_bar_ipmax': None,\n",
       " 'cpf_ne_yag_bar_ipmax': None,\n",
       " 'cpf_ngreenwald_ipmax': None,\n",
       " 'cpf_ngreenwaldratio_ipmax': None,\n",
       " 'cpf_o2ratio': None,\n",
       " 'cpf_objective': None,\n",
       " 'cpf_pe0_ipmax': None,\n",
       " 'cpf_pe0_max': None,\n",
       " 'cpf_pe0_truby': None,\n",
       " 'cpf_pe0ruby': None,\n",
       " 'cpf_pic': None,\n",
       " 'cpf_pnbi_ipmax': None,\n",
       " 'cpf_pnbi_ipmax_ss': None,\n",
       " 'cpf_pnbi_ipmax_sw': None,\n",
       " 'cpf_pnbi_max': None,\n",
       " 'cpf_pnbi_max_ss': None,\n",
       " 'cpf_pnbi_max_sw': None,\n",
       " 'cpf_pnbi_truby': None,\n",
       " 'cpf_pnbi_truby_ss': None,\n",
       " 'cpf_pnbi_truby_sw': None,\n",
       " 'cpf_pohm_ipmax': None,\n",
       " 'cpf_pohm_max': None,\n",
       " 'cpf_pohm_truby': None,\n",
       " 'cpf_postshot': None,\n",
       " 'cpf_prad_ipmax': None,\n",
       " 'cpf_prad_max': None,\n",
       " 'cpf_prad_truby': None,\n",
       " 'cpf_pradne2': None,\n",
       " 'cpf_preshot': None,\n",
       " 'cpf_program': None,\n",
       " 'cpf_pulno': None,\n",
       " 'cpf_q95_ipmax': None,\n",
       " 'cpf_q95_min': None,\n",
       " 'cpf_q95_truby': None,\n",
       " 'cpf_reference': None,\n",
       " 'cpf_rgeo_ipmax': None,\n",
       " 'cpf_rgeo_max': None,\n",
       " 'cpf_rgeo_truby': None,\n",
       " 'cpf_rinner_da': None,\n",
       " 'cpf_rinner_efit': None,\n",
       " 'cpf_rmag_efit': None,\n",
       " 'cpf_router_da': None,\n",
       " 'cpf_router_efit': None,\n",
       " 'cpf_sarea_ipmax': None,\n",
       " 'cpf_sarea_max': None,\n",
       " 'cpf_sarea_truby': None,\n",
       " 'cpf_sl': None,\n",
       " 'cpf_summary': None,\n",
       " 'cpf_tamin_max': None,\n",
       " 'cpf_tarea_max': None,\n",
       " 'cpf_tautot_ipmax': None,\n",
       " 'cpf_tautot_max': None,\n",
       " 'cpf_tautot_truby': None,\n",
       " 'cpf_tbepmhd_max': None,\n",
       " 'cpf_tbetmhd_max': None,\n",
       " 'cpf_tbt_max': None,\n",
       " 'cpf_tdwmhd_max': None,\n",
       " 'cpf_te0_ipmax': None,\n",
       " 'cpf_te0_max': None,\n",
       " 'cpf_te0_truby': None,\n",
       " 'cpf_te0ratio_ipmax': None,\n",
       " 'cpf_te0ruby': None,\n",
       " 'cpf_te_yag_bar_ipmax': None,\n",
       " 'cpf_tend': None,\n",
       " 'cpf_tend_ibgas': None,\n",
       " 'cpf_tend_nbi': None,\n",
       " 'cpf_tend_nbi_ss': None,\n",
       " 'cpf_tend_nbi_sw': None,\n",
       " 'cpf_term_code': None,\n",
       " 'cpf_tftend': None,\n",
       " 'cpf_tftstart': None,\n",
       " 'cpf_tipmax': None,\n",
       " 'cpf_tkappa_max': None,\n",
       " 'cpf_tli_2_max': None,\n",
       " 'cpf_tli_3_max': None,\n",
       " 'cpf_tndl_co2_max': None,\n",
       " 'cpf_tne0_max': None,\n",
       " 'cpf_tpe0_max': None,\n",
       " 'cpf_tpnbi_max': None,\n",
       " 'cpf_tpnbi_max_ss': None,\n",
       " 'cpf_tpnbi_max_sw': None,\n",
       " 'cpf_tpohm_max': None,\n",
       " 'cpf_tprad_max': None,\n",
       " 'cpf_tq95_min': None,\n",
       " 'cpf_trgeo_max': None,\n",
       " 'cpf_truby': None,\n",
       " 'cpf_tsarea_max': None,\n",
       " 'cpf_tstart': None,\n",
       " 'cpf_tstart_ibgas': None,\n",
       " 'cpf_tstart_nbi': None,\n",
       " 'cpf_tstart_nbi_ss': None,\n",
       " 'cpf_tstart_nbi_sw': None,\n",
       " 'cpf_ttautot_max': None,\n",
       " 'cpf_tte0_max': None,\n",
       " 'cpf_tvol_max': None,\n",
       " 'cpf_twmhd_max': None,\n",
       " 'cpf_tzeff_max': None,\n",
       " 'cpf_useful': None,\n",
       " 'cpf_vol_ipmax': None,\n",
       " 'cpf_vol_max': None,\n",
       " 'cpf_vol_truby': None,\n",
       " 'cpf_wmhd_ipmax': None,\n",
       " 'cpf_wmhd_max': None,\n",
       " 'cpf_wmhd_truby': None,\n",
       " 'cpf_zeff_ipmax': None,\n",
       " 'cpf_zeff_max': None,\n",
       " 'cpf_zeff_truby': None,\n",
       " 'cpf_zmag_efit': None}"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result['items'][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a7a8cfac",
   "metadata": {},
   "source": [
    "Each item in the list is a json object. This contains the meta-data items that corresponded to our query. In this case, each item contains information about a different MAST shot. Each item has lots of information about different shots, for example the shot ID, the campaign the shot was part of, the pre- and post-shot description by investigators.\n",
    "\n",
    "For more information on the what's returned by the API you can look at the endpoint documentation:\n",
    "\n",
    "https://mast-app.site/redoc\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f19e602d",
   "metadata": {},
   "source": [
    "Of course, we can read all this JSON data directly into common python data analysis packages, for example, we can create a `pandas` dataframe directly from the endpoint data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "66c264c5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>shot_id</th>\n",
       "      <th>uuid</th>\n",
       "      <th>url</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>preshot_description</th>\n",
       "      <th>postshot_description</th>\n",
       "      <th>campaign</th>\n",
       "      <th>reference_shot</th>\n",
       "      <th>scenario</th>\n",
       "      <th>heating</th>\n",
       "      <th>...</th>\n",
       "      <th>cpf_vol_ipmax</th>\n",
       "      <th>cpf_vol_max</th>\n",
       "      <th>cpf_vol_truby</th>\n",
       "      <th>cpf_wmhd_ipmax</th>\n",
       "      <th>cpf_wmhd_max</th>\n",
       "      <th>cpf_wmhd_truby</th>\n",
       "      <th>cpf_zeff_ipmax</th>\n",
       "      <th>cpf_zeff_max</th>\n",
       "      <th>cpf_zeff_truby</th>\n",
       "      <th>cpf_zmag_efit</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>15085</td>\n",
       "      <td>0bb8ea1b-954b-5cce-8b2d-df40a0703b2f</td>\n",
       "      <td>s3://mast/level1/shots/15085.zarr</td>\n",
       "      <td>2006-04-25T13:48:00</td>\n",
       "      <td>\\nReplace ccbv connector. repeat.\\n</td>\n",
       "      <td>\\nRadial control unchanged.  Zip/FA jump less ...</td>\n",
       "      <td>M6</td>\n",
       "      <td>15084.0</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>15086</td>\n",
       "      <td>2ac3c2b7-0444-5f3b-86a9-38ef6bb3c160</td>\n",
       "      <td>s3://mast/level1/shots/15086.zarr</td>\n",
       "      <td>2006-04-25T14:15:00</td>\n",
       "      <td>\\nAdd another 5cm to Drref during the ramp. Re...</td>\n",
       "      <td>\\nEarly termination, at 180ms.\\n</td>\n",
       "      <td>M6</td>\n",
       "      <td>15085.0</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>15087</td>\n",
       "      <td>9867930b-af84-5866-a62c-9fe77d9a3def</td>\n",
       "      <td>s3://mast/level1/shots/15087.zarr</td>\n",
       "      <td>2006-04-25T14:34:00</td>\n",
       "      <td>\\nRepeat, with small change to IDIV.\\n</td>\n",
       "      <td>\\nEarly termination at 200ms.\\n</td>\n",
       "      <td>M6</td>\n",
       "      <td>15086.0</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>15088</td>\n",
       "      <td>f3f7ffe4-bfd1-5b8d-8856-9d7e4377724b</td>\n",
       "      <td>s3://mast/level1/shots/15088.zarr</td>\n",
       "      <td>2006-04-25T14:52:00</td>\n",
       "      <td>\\nRestore shot 15085.\\n</td>\n",
       "      <td>\\nOK.\\n</td>\n",
       "      <td>M6</td>\n",
       "      <td>15085.0</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>15089</td>\n",
       "      <td>1adae615-46c4-5262-a49d-b25f707b0857</td>\n",
       "      <td>s3://mast/level1/shots/15089.zarr</td>\n",
       "      <td>2006-04-25T15:17:00</td>\n",
       "      <td>\\nRepeat, with IDIV, BV tweaked to match last ...</td>\n",
       "      <td>\\nTerminated early.\\n</td>\n",
       "      <td>M6</td>\n",
       "      <td>15088.0</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 282 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   shot_id                                  uuid  \\\n",
       "0    15085  0bb8ea1b-954b-5cce-8b2d-df40a0703b2f   \n",
       "1    15086  2ac3c2b7-0444-5f3b-86a9-38ef6bb3c160   \n",
       "2    15087  9867930b-af84-5866-a62c-9fe77d9a3def   \n",
       "3    15088  f3f7ffe4-bfd1-5b8d-8856-9d7e4377724b   \n",
       "4    15089  1adae615-46c4-5262-a49d-b25f707b0857   \n",
       "\n",
       "                                 url            timestamp  \\\n",
       "0  s3://mast/level1/shots/15085.zarr  2006-04-25T13:48:00   \n",
       "1  s3://mast/level1/shots/15086.zarr  2006-04-25T14:15:00   \n",
       "2  s3://mast/level1/shots/15087.zarr  2006-04-25T14:34:00   \n",
       "3  s3://mast/level1/shots/15088.zarr  2006-04-25T14:52:00   \n",
       "4  s3://mast/level1/shots/15089.zarr  2006-04-25T15:17:00   \n",
       "\n",
       "                                 preshot_description  \\\n",
       "0                \\nReplace ccbv connector. repeat.\\n   \n",
       "1  \\nAdd another 5cm to Drref during the ramp. Re...   \n",
       "2             \\nRepeat, with small change to IDIV.\\n   \n",
       "3                            \\nRestore shot 15085.\\n   \n",
       "4  \\nRepeat, with IDIV, BV tweaked to match last ...   \n",
       "\n",
       "                                postshot_description campaign  reference_shot  \\\n",
       "0  \\nRadial control unchanged.  Zip/FA jump less ...       M6         15084.0   \n",
       "1                   \\nEarly termination, at 180ms.\\n       M6         15085.0   \n",
       "2                    \\nEarly termination at 200ms.\\n       M6         15086.0   \n",
       "3                                            \\nOK.\\n       M6         15085.0   \n",
       "4                              \\nTerminated early.\\n       M6         15088.0   \n",
       "\n",
       "  scenario heating  ...  cpf_vol_ipmax cpf_vol_max cpf_vol_truby  \\\n",
       "0     None    None  ...           None        None          None   \n",
       "1     None    None  ...           None        None          None   \n",
       "2     None    None  ...           None        None          None   \n",
       "3     None    None  ...           None        None          None   \n",
       "4     None    None  ...           None        None          None   \n",
       "\n",
       "  cpf_wmhd_ipmax cpf_wmhd_max cpf_wmhd_truby cpf_zeff_ipmax cpf_zeff_max  \\\n",
       "0           None         None           None           None         None   \n",
       "1           None         None           None           None         None   \n",
       "2           None         None           None           None         None   \n",
       "3           None         None           None           None         None   \n",
       "4           None         None           None           None         None   \n",
       "\n",
       "  cpf_zeff_truby cpf_zmag_efit  \n",
       "0           None          None  \n",
       "1           None          None  \n",
       "2           None          None  \n",
       "3           None          None  \n",
       "4           None          None  \n",
       "\n",
       "[5 rows x 282 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_json(f'{API_URL}/shots')['items']\n",
    "df = pd.DataFrame(df.to_list())\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9de70947",
   "metadata": {},
   "source": [
    "### Searching & Filtering Data\n",
    "\n",
    "All REST API endpoints can take query parameters to filter the data returned. For example, we can return all shots for the `M9` campaign by using the approrpiate query string.\n",
    "\n",
    "For example, we can query for everything from the `M9` campaign by adding `?campaign=M9` to our query string."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "ed99237a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>shot_id</th>\n",
       "      <th>uuid</th>\n",
       "      <th>url</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>preshot_description</th>\n",
       "      <th>postshot_description</th>\n",
       "      <th>campaign</th>\n",
       "      <th>reference_shot</th>\n",
       "      <th>scenario</th>\n",
       "      <th>heating</th>\n",
       "      <th>...</th>\n",
       "      <th>cpf_vol_ipmax</th>\n",
       "      <th>cpf_vol_max</th>\n",
       "      <th>cpf_vol_truby</th>\n",
       "      <th>cpf_wmhd_ipmax</th>\n",
       "      <th>cpf_wmhd_max</th>\n",
       "      <th>cpf_wmhd_truby</th>\n",
       "      <th>cpf_zeff_ipmax</th>\n",
       "      <th>cpf_zeff_max</th>\n",
       "      <th>cpf_zeff_truby</th>\n",
       "      <th>cpf_zmag_efit</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>15085</td>\n",
       "      <td>0bb8ea1b-954b-5cce-8b2d-df40a0703b2f</td>\n",
       "      <td>s3://mast/level1/shots/15085.zarr</td>\n",
       "      <td>2006-04-25T13:48:00</td>\n",
       "      <td>\\nReplace ccbv connector. repeat.\\n</td>\n",
       "      <td>\\nRadial control unchanged.  Zip/FA jump less ...</td>\n",
       "      <td>M6</td>\n",
       "      <td>15084.0</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>15086</td>\n",
       "      <td>2ac3c2b7-0444-5f3b-86a9-38ef6bb3c160</td>\n",
       "      <td>s3://mast/level1/shots/15086.zarr</td>\n",
       "      <td>2006-04-25T14:15:00</td>\n",
       "      <td>\\nAdd another 5cm to Drref during the ramp. Re...</td>\n",
       "      <td>\\nEarly termination, at 180ms.\\n</td>\n",
       "      <td>M6</td>\n",
       "      <td>15085.0</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>15087</td>\n",
       "      <td>9867930b-af84-5866-a62c-9fe77d9a3def</td>\n",
       "      <td>s3://mast/level1/shots/15087.zarr</td>\n",
       "      <td>2006-04-25T14:34:00</td>\n",
       "      <td>\\nRepeat, with small change to IDIV.\\n</td>\n",
       "      <td>\\nEarly termination at 200ms.\\n</td>\n",
       "      <td>M6</td>\n",
       "      <td>15086.0</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>15088</td>\n",
       "      <td>f3f7ffe4-bfd1-5b8d-8856-9d7e4377724b</td>\n",
       "      <td>s3://mast/level1/shots/15088.zarr</td>\n",
       "      <td>2006-04-25T14:52:00</td>\n",
       "      <td>\\nRestore shot 15085.\\n</td>\n",
       "      <td>\\nOK.\\n</td>\n",
       "      <td>M6</td>\n",
       "      <td>15085.0</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>15089</td>\n",
       "      <td>1adae615-46c4-5262-a49d-b25f707b0857</td>\n",
       "      <td>s3://mast/level1/shots/15089.zarr</td>\n",
       "      <td>2006-04-25T15:17:00</td>\n",
       "      <td>\\nRepeat, with IDIV, BV tweaked to match last ...</td>\n",
       "      <td>\\nTerminated early.\\n</td>\n",
       "      <td>M6</td>\n",
       "      <td>15088.0</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>...</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 282 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   shot_id                                  uuid  \\\n",
       "0    15085  0bb8ea1b-954b-5cce-8b2d-df40a0703b2f   \n",
       "1    15086  2ac3c2b7-0444-5f3b-86a9-38ef6bb3c160   \n",
       "2    15087  9867930b-af84-5866-a62c-9fe77d9a3def   \n",
       "3    15088  f3f7ffe4-bfd1-5b8d-8856-9d7e4377724b   \n",
       "4    15089  1adae615-46c4-5262-a49d-b25f707b0857   \n",
       "\n",
       "                                 url            timestamp  \\\n",
       "0  s3://mast/level1/shots/15085.zarr  2006-04-25T13:48:00   \n",
       "1  s3://mast/level1/shots/15086.zarr  2006-04-25T14:15:00   \n",
       "2  s3://mast/level1/shots/15087.zarr  2006-04-25T14:34:00   \n",
       "3  s3://mast/level1/shots/15088.zarr  2006-04-25T14:52:00   \n",
       "4  s3://mast/level1/shots/15089.zarr  2006-04-25T15:17:00   \n",
       "\n",
       "                                 preshot_description  \\\n",
       "0                \\nReplace ccbv connector. repeat.\\n   \n",
       "1  \\nAdd another 5cm to Drref during the ramp. Re...   \n",
       "2             \\nRepeat, with small change to IDIV.\\n   \n",
       "3                            \\nRestore shot 15085.\\n   \n",
       "4  \\nRepeat, with IDIV, BV tweaked to match last ...   \n",
       "\n",
       "                                postshot_description campaign  reference_shot  \\\n",
       "0  \\nRadial control unchanged.  Zip/FA jump less ...       M6         15084.0   \n",
       "1                   \\nEarly termination, at 180ms.\\n       M6         15085.0   \n",
       "2                    \\nEarly termination at 200ms.\\n       M6         15086.0   \n",
       "3                                            \\nOK.\\n       M6         15085.0   \n",
       "4                              \\nTerminated early.\\n       M6         15088.0   \n",
       "\n",
       "  scenario heating  ...  cpf_vol_ipmax cpf_vol_max cpf_vol_truby  \\\n",
       "0     None    None  ...           None        None          None   \n",
       "1     None    None  ...           None        None          None   \n",
       "2     None    None  ...           None        None          None   \n",
       "3     None    None  ...           None        None          None   \n",
       "4     None    None  ...           None        None          None   \n",
       "\n",
       "  cpf_wmhd_ipmax cpf_wmhd_max cpf_wmhd_truby cpf_zeff_ipmax cpf_zeff_max  \\\n",
       "0           None         None           None           None         None   \n",
       "1           None         None           None           None         None   \n",
       "2           None         None           None           None         None   \n",
       "3           None         None           None           None         None   \n",
       "4           None         None           None           None         None   \n",
       "\n",
       "  cpf_zeff_truby cpf_zmag_efit  \n",
       "0           None          None  \n",
       "1           None          None  \n",
       "2           None          None  \n",
       "3           None          None  \n",
       "4           None          None  \n",
       "\n",
       "[5 rows x 282 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_json(f\"{API_URL}/shots?campaign=M9\")['items']\n",
    "df = pd.DataFrame(df.to_list())\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2e54b05-98f5-4ff2-a2e9-be3e356e0c3a",
   "metadata": {},
   "source": [
    "### Pagination \n",
    "The REST API responses are _paginated_, meaning that only a subset of the full items are returned with each query. Pagination is used to limit the total number of requests made by each user to prevent any single user overloading the server with huge data requests.\n",
    "\n",
    "Pagination information is included in the response and corresponds to [RFC 8288](https://datatracker.ietf.org/doc/html/rfc8288). The response contains cursors at the bottom of the response:\n",
    "\n",
    " - `current_page` - this is the cursor relating to the current page of results.\n",
    " - `next_page` - this is the cursor for the next page of results.\n",
    " - `previous_page` - this is the cursor for the previous page of results, will result in `null` if on the first page.\n",
    "\n",
    "You can control the pagination using the following options as query arguments:\n",
    " - `cursor=xx` to set the cursor for the displayed set of results\n",
    " - `size=xx` to set the number of results returned by each page."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "80d840d5-ccf4-4ed2-9fe4-33798f2ae62f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Current page cursor Pmk6MzAxMTE%3D\n",
      "Next page cursor Pmk6MzAxMTM%3D\n",
      "Previous page cursor PGk6MzAxMTI%3D\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>shot_id</th>\n",
       "      <th>uuid</th>\n",
       "      <th>url</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>preshot_description</th>\n",
       "      <th>postshot_description</th>\n",
       "      <th>campaign</th>\n",
       "      <th>reference_shot</th>\n",
       "      <th>scenario</th>\n",
       "      <th>heating</th>\n",
       "      <th>...</th>\n",
       "      <th>cpf_vol_ipmax</th>\n",
       "      <th>cpf_vol_max</th>\n",
       "      <th>cpf_vol_truby</th>\n",
       "      <th>cpf_wmhd_ipmax</th>\n",
       "      <th>cpf_wmhd_max</th>\n",
       "      <th>cpf_wmhd_truby</th>\n",
       "      <th>cpf_zeff_ipmax</th>\n",
       "      <th>cpf_zeff_max</th>\n",
       "      <th>cpf_zeff_truby</th>\n",
       "      <th>cpf_zmag_efit</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>30112</td>\n",
       "      <td>eefe2467-69b0-5fe7-9863-e21289daa135</td>\n",
       "      <td>s3://mast/level1/shots/30112.zarr</td>\n",
       "      <td>2013-09-09T11:42:00</td>\n",
       "      <td>\\nReload 30110 - reduce IP flat top to 750 KA ...</td>\n",
       "      <td>\\nPIC happy with data.\\n</td>\n",
       "      <td>M9</td>\n",
       "      <td>30110</td>\n",
       "      <td>4</td>\n",
       "      <td>SS Beam</td>\n",
       "      <td>...</td>\n",
       "      <td>7.845118</td>\n",
       "      <td>8.841636</td>\n",
       "      <td>0.0</td>\n",
       "      <td>65898.305250</td>\n",
       "      <td>82056.664</td>\n",
       "      <td>0.0</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>0.047497</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>30113</td>\n",
       "      <td>742c78fa-3811-5404-93b3-5ab419d330bd</td>\n",
       "      <td>s3://mast/level1/shots/30113.zarr</td>\n",
       "      <td>2013-09-09T12:00:00</td>\n",
       "      <td>\\nRestore 30111\\n</td>\n",
       "      <td>\\nShot OK for programme. Two good beams. Big l...</td>\n",
       "      <td>M9</td>\n",
       "      <td>30111</td>\n",
       "      <td>3</td>\n",
       "      <td>2 Beams,SS Beam,SW Beam</td>\n",
       "      <td>...</td>\n",
       "      <td>8.391672</td>\n",
       "      <td>8.426641</td>\n",
       "      <td>0.0</td>\n",
       "      <td>43127.703046</td>\n",
       "      <td>60260.926</td>\n",
       "      <td>0.0</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>0.042220</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2 rows × 282 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   shot_id                                  uuid  \\\n",
       "0    30112  eefe2467-69b0-5fe7-9863-e21289daa135   \n",
       "1    30113  742c78fa-3811-5404-93b3-5ab419d330bd   \n",
       "\n",
       "                                 url            timestamp  \\\n",
       "0  s3://mast/level1/shots/30112.zarr  2013-09-09T11:42:00   \n",
       "1  s3://mast/level1/shots/30113.zarr  2013-09-09T12:00:00   \n",
       "\n",
       "                                 preshot_description  \\\n",
       "0  \\nReload 30110 - reduce IP flat top to 750 KA ...   \n",
       "1                                  \\nRestore 30111\\n   \n",
       "\n",
       "                                postshot_description campaign  reference_shot  \\\n",
       "0                           \\nPIC happy with data.\\n       M9           30110   \n",
       "1  \\nShot OK for programme. Two good beams. Big l...       M9           30111   \n",
       "\n",
       "   scenario                  heating  ...  cpf_vol_ipmax  cpf_vol_max  \\\n",
       "0         4                  SS Beam  ...       7.845118     8.841636   \n",
       "1         3  2 Beams,SS Beam,SW Beam  ...       8.391672     8.426641   \n",
       "\n",
       "  cpf_vol_truby cpf_wmhd_ipmax cpf_wmhd_max cpf_wmhd_truby cpf_zeff_ipmax  \\\n",
       "0           0.0   65898.305250    82056.664            0.0           None   \n",
       "1           0.0   43127.703046    60260.926            0.0           None   \n",
       "\n",
       "  cpf_zeff_max cpf_zeff_truby cpf_zmag_efit  \n",
       "0         None           None      0.047497  \n",
       "1         None           None      0.042220  \n",
       "\n",
       "[2 rows x 282 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response = requests.get(f'{API_URL}/shots?size=2&cursor=Pmk6MzAxMTE%3D')\n",
    "result = response.json()\n",
    "headers = response.headers\n",
    "\n",
    "print(\"Current page cursor\", result['current_page'])\n",
    "print(\"Next page cursor\", result['next_page'])\n",
    "print(\"Previous page cursor\", result['previous_page'])\n",
    "\n",
    "df = pd.DataFrame(result['items'])\n",
    "df"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "mast",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}