blog.natfan.io

Rants and ravings from a techy brit.
(Now hosted on DigitalOcean!)
Dark Mode?

Making a Cross Domain APIs (with NamelessMC and Lumen)

Posted 2 months ago.

Hi all, back again with some more stuff on APIs. Gosh, I love APIs. They're an amazing way of sharing data between services, and making the Internet more open. But how do you share data from one domain to another? That question might sound easy if you know who's making the request, but what if you don't?

For reference, I'm using the super awesome framework Lumen to write this API, so I'll use code snippets from that to produce the example code used in this blog post.

My real world example is api.colossalmc.net, an API that I'm writing for my newest venture, a Zombie survival experience in Minecraft. In that project I'm a bit of a Jack of all Trades. I'm a member of the leadership team, a developer and an interim community manager. In my opinion APIs are really a mix of those last two jobs. You can manage your community better by having a good API. You can let your players make amazing tools out of your open data, and it's actually how I got started with web development! I was asked by a player to make a leaderboard, so that people can see their stats and try to get higher up than others. A relatively simple task, however I wanted to take this opportunity to start work on an API for the whole network. The problem that I recently finished tackling, and one that I want to talk about in this post, in authentication. Or, more accurately, validation of authentication.

So we have a website for the server, where users have to sign up by running /register in-game. This gives them a URL that they can access to create an account that is automagically tied to their in-game account. Pretty cool, right? I wish I could take credit but it's all done by the amazing people who made NamelessMC, really a brilliant bunch. So, we have an authoritative source of who someone is, how do we transfer that elsewhere?

For reasons that I won't really go into, I'm having to host this API on another box. This means that I can't just get the data from the user's profile directly, I need a way to transfer it to my API subdomain on a different server (in a different part of the planet, in fact). My first thought was to ask the player for their account's UUID in the form of a POST request to the API server, then give them a key.

See below for an example:

// when we make a POST request to api.colossalmc.net/key
$router->post('/key', function(Request $request) {
  // initalize the error data with our default value (success)
  $error = false;
  $error_msg = null;
  $error_code = 200;

  // get all of the JSON data that was sent to us in the request
  $json = $request->json()->all();
  $rules = [
    // make sure that the UUID exists in the request, is a valid UUID
    // and is relating to a user in our database
    'uuid' => 'required|uuid|exists:users',
  ];
  $validator = Validator::make($json, $rules);
  if ($validator->fails()) {
    // if the check fails, report back why with a 400 (BadRequest) error
    return response()->json($validator->messages(), 400);
  }

  // generate a key for the user, attach it to their account and save the database
  $user = User::find($uuid);
  $key = Str::random(32);
  $user->key = $key;
  $user->save();

  // return the base output to the user with the errors (if there were any)
  $output = array(
    "error" => $error,
    "error_msg" => $error_msg,
  );
  // if there weren't any errors, return the data
  if (!$error) {
    $output["data"] = array(
    "user" => $user,
    "key" => $key,
    );
  }
  return response($output, $error_code);
});

Basically, get what the user gave us, generate a key and pass it back to them. That works fine, but what if I'm sneaky and instead of giving the endpoint my UUID, we give them someone else's? Crap, our system isn't secure. Also, give that all of Minecraft's UUIDs are publicly available, it's not like impersonation would be hard for a bad actor to implement!

Okay, back to the drawing board slightly. What if I instead try the following:

// show a view when the user visits '/key' instead, and move the
// '/key' POST request from earlier to POST '/key/confirm'
$router->get('/key', function(Request $request) {
  $view = View::make('key');
  return response($view);
});
<!DOCTYPE html>
<html>
  <head>
  <head>
  <body>
    <!-- use this as a placeholder -->
    <p id="data">{"error":false,"error_msg":null,"data":"pending"}</p>
    <script>
      var errorMessage = null;

      <!-- connect to the authoritative source -->
      var result = fetch("https://colossalmc.net/queries/me", {
        <!-- make sure to use the credentials (stored as cookies) -->
        credentials: "include",
      }).then(response => {
        <!-- translate the JSON data into a JS object -->
        return response.json()
      }).then(json => {
        let payload;
        <!-- if we're getting an error, just set the UUID to be a blank UUID -->
        if (json.error) {
          payload = {
            uuid: "00000000-0000-0000-0000-000000000000",
          }
        } else {
          <!-- if we're getting valid data, set the UUID to be the one that
          we just received from the authoritative source -->
          payload = {
            uuid: json.data.uuid,
          }
        }
        <!-- now, go back to the key generator code with the UUID
        that we got from the authoritative source -->
        return fetch(window.location.origin + "/v1/key/confirm", {
          method: "POST",
          body: JSON.stringify(payload),
        })
      }).then(response => {
        <!-- translate the JSON data into a JS object -->
        return response.json()
      }).catch(error => errorMessage = error);

      result.then(r => {
        <!-- as we were doing this in text/html, return the data as text/json -->
        let url = window.location.origin + "/v1/showJSON?data=" + JSON.stringify(r)
        window.location = url;
      });
    </script>
  <body>
<html>

The interesting bit of this is done in https://colossalmc.net/queries/me, the NamelessMC site which already has a framework for writing utilities called "Queries". I just copied one and edited it slightly.

// only allow access from "api.colossalmc.net"
header("Access-Control-Allow-Origin: https://api.colossalmc.net");
// allow credential sharing (via cookies!)
header("Access-Control-Allow-Credentials: true");

// the stuff we want to output
$output = array();

// are we logged in?
if(!$user->isLoggedIn()) {
  // nope? okay, return an error
  $output['error'] = true;
  $output['error_msg'] = "NotAuthenticated: User not logged in.";
  http_response_code(401);
} else {
  // we are logged in! convert the UUID to be one with dashes for readability
  $uuid = $user->data()->uuid;
  $uuid = substr($uuid, 0, 8) . '-' . substr($uuid, 8, 4) . '-' . substr($uuid, 12, 4) . '-' . substr($uuid, 16, 4)  . '-' . substr($uuid, 20);
  $user->data()->uuid = $uuid;

  // add the data to the output
  $output["error"] = false;
  $output["data"] = get_object_vars($user->data());
}

// return the data as pretty JSON
echo json_encode($output);
?>

Okay awesome, we're getting somewhere! We've managed to get the logged in user from NamelessMC, and then return that data to the API key generator. We're all good, except for one critical flaw. What happens if someone just decided to send the data that NamelessMC sends the API themselves, and edits the UUID to be whatever they want? Drat! We're back at square one! Or are we?

Now that we've done all the heavy lifting, it shouldn't be too hard to actually get this secured. We know that the UUID is going to be the same on both sides of the API, so how do we confirm that it hasn't been tampered with? We could use the Referer and Origin headers, but those are set by the client and can be spoofed. We need a way for these two distinct servers to be able to validate the data. But how, you ask? HMAC!

What we want to do is add in a super duper secret key that is the same on both servers. We don't want to transmit that key in it's raw form because if someone knows what it is, they could make their own requests seem legitimate. That's where HMAC comes in.

Basically, HMAC is a method of hashing data with a secret key and a UNIX timestamp, meaning that you know that the data hasn't been tampered with or suffered a timing attack. HMACs are also the same every time they're generated, as long as they're generated within a similar timeframe. All we need to do is add the following code to the NamelessMC query:

// this secret is the same on both NamelessMC and the API to ensure integrity.
$secret = "oh boy this key is so secret";
$hmac = hash_hmac("sha256", $user->data()->uuid, $secret);
$output["hmac"] = $hmac;

Now we're returning the HMAC to the Javascript portion of the code, we need to have the JS bit report that to the Lumen backend:

// dummy payload if the request fails
payload = {
  uuid: "00000000-0000-0000-0000-000000000000",
  hmac: "0000000000000000000000000000000000000000000000000000000000000000",
}

// real payload if the request succeeds
payload = {
  uuid: json.data.uuid,
  hmac: json.hmac,
}

And finally we need to update the Lumen backend to generate a HMAC based on the data that we have (UUID and secret) and compare it with the data that the authoritative source gave us:

// this secret is the same on both NamelessMC and the API to ensure integrity.
  $secret = "oh boy this key is so secret"
  $json = $request->json()->all();
  $rules = [
    'uuid' => 'required|uuid|exists:users',
    // ensure that the HMAC is the right size (64 characters)
    'hmac' => 'required|size:64',
  ];
  ...
  $uuid = $request->json('uuid');
  // get the HMAC submitted by the client
  $their_hmac = $request->json('hmac');
  // generate a HMAC based on the data we have (UUID and secret)
  $our_hmac = hash_hmac("sha256", $uuid, $secret);
  // compare the hashes
  $hash_equals = hash_equals($their_hmac, $our_hmac);

  // error out if the HMAC they provided doesn't equal the one we calculated!
  if (!$hash_equals) {
    $error = true;
    $error_msg = "Unauthenticated: HMAC is invalid";
    $error_code = 401;
  }

And that's it! Let's recap on what we've done, as it might have gotten a bit complex there:

  1. The user visits https://api.colossalmc.net/key, a page with JS on it.
  2. The JS queries the NamelessMC instance, the authoritative source.
  3. The NamelessMC instance returns the logged in user and a secure key that is generated based on the time that the request was made, the UUID of the logged in user and the secret key that we have stored on the server.
  4. The JS page then returns that data it received from NamelessMC to the Lumen backend.
  5. The Lumen backend gets the UUID and HMAC that was generated before, and tries to generate it's own HMAC based on the UUID it was provided. Remember, it should be the same as the UUID and secret key are the same on both sides.
  6. If the Lumen backend confirms that the data is valid, it generates a key and returns it to the JS page.
  7. The JS page just returns that data to the user as pretty JSON.
  8. The user can now authenticate with the API without having to make a new account.

Pretty neat, huh? :)

Hope you enjoyed! If you want to try out my newest venture, you can visit colossalmc.net to get more information and connect with play.colossalmc.net.

See you next time,

-nat