Part #1: Building Serverless File Hosting Web Application (Back-end)
This is the first part of a series where we will be looking into building secure serverless file hosting web application. This part will focus on building GraphQL API for managing user’s content metadata and also providing a secure way of accessing users’ content through CloudFront.
Solution Architecture
The overall solution architecture looks like below:
Let’s go through steps in detail:
1 — Web app host IP address gets resolved (DNS hosted in Route 53 public zone)
2 — Web app requires user to get authenticated against Cognito
3 — Web app sends request to AppSync GraphQL endpoint to retrieve user’s objects metadata
4 — GraphQL query retrieves user’s objects metadata from Dynamo DB
5 — AppSync calls Lambda resolver with unsigned object URL
6 — Lambda resolver retrieves CloudFront Signing Key Pair ID and CloudFront Private Key from Systems Manager Parameters Store and signs provided URL
7 — Web App uses signed URL to retrieve user’s object from CDN
8 — Authenticated Web App user sends PUT request for placing new objects to S3
Implementation
Back-end implementation focuses on steps 4, 5, 6, 7 of the architecture above, steps 1–3 and 8 will be discussed in the next part of this series.
CloudFront Signing Key Pair
We will start with creating a signing key pair for CloudFront URLs. As an AWS account owner navigate to “My Security Credentials” > “CloudFront key pairs” and create a new one. Load Key Pair ID and Private Key to AWS System Manager Parameter Store, we will be using them later:
# aws ssm put-parameter --name sign_cdn.key --value "-----BEGIN RSA PRIVATE KEY-----
...
-----END RSA PRIVATE KEY-----" --type String --region us-east-1# aws ssm put-parameter --name sign_cdn.key_id --value APKAIMUSFQLY67DRKW2A --type String --region us-east-1
S3 Bucket as a CloudFront Origin
Simply create a private S3 bucket for CloudFront distribution origin, this is where users’ files are going to be stored:
Public Route 53 Zone to host DNS records for CDN and AppSync CloudFront
Navigate to AWS Route 53 and create a public zone to host DNS records for domain of your choice, make sure that Route 53 DNS servers are registered in your Domain Registrar DNS config.
SSL certificate for CDN and AppSync CloudFront distributions
Navigate to AWS Certificate Manager and request a new public SSL certificates for CDN and AppSync endpoints, choose DNS validation, after validation is complete you should see status of the certificates as “Issued”.
CloudFront distribution for private content
Create CloudFront Distribution for private content
Select an SSL certificate that was created earlier to be used for a new distribution:
It will take some time for distribution to get deployed to CloudFront edge locations:
Once distribution is created, create a Route 53 custom domain name entry pointing to a distribution domain name:
Cognito User Pool
As an identity provider we are going to use AWS Cognito. Create a Cognito User Pool with Application Client settings as below:
Configure Cognito App Client with supported OAuth flows, Identity Providers and allowed Callback URLs, we will require this to test our back-end:
We will also require a domain name for getting an identity token while testing
Sign CDN URLs Lambda function
Create a lambda function which AppSync is going to use as a resolver for signing CDN URLs using Private Key and Key Pair ID from AWS System Manager Parameter Store.
var aws = require('aws-sdk');
var ssm = new aws.SSM();exports.handler = async (event) => {
let parameters,
params = {
Names: [
'sign_cdn.key',
'sign_cdn.key_id'
]
},
options = {
url: event.url,
expires: Math.floor(new Date() / 1000) + 3600
},
keyPairId,
privateKey;try {
parameters = await ssm.getParameters(params).promise()
.then(function(data){
return data.Parameters;
});
}catch(error){
console.log('There was an issue while getting signing key and key id:',error);
}parameters.map((parameter) => {
(parameter.Name == 'sign_cdn.key') ? privateKey = parameter.Value : false;
(parameter.Name == 'sign_cdn.key_id') ? keyPairId = parameter.Value : false;
});let cloudFrontSigner = new aws.CloudFront.Signer(keyPairId, privateKey);
return cloudFrontSigner.getSignedUrl(options);
};
AppSync API
Create an AppSync API with the schema as below:
type Mutation {
putObject(objectId: String!, comment: String): Object
deleteObject(objectId: String!): Object
}type Object {
objectId: String
url: String
comment: String
}type Query {
getObject(objectId: String!): Object
}schema {
query: Query
mutation: Mutation
}
Select AWS Cognito as an Authorization type for AppSync API:
As an AppSync datasource we will create a single Dynamo DB table with a composite primary key where hash key is a userId and range key is an objectId, this structure will allow to store metadata for objects that are owned by a particular userId:
Let’s define putObject mutation resolver:
{
"version" : "2017-02-28",
"operation" : "PutItem",
"key" : {
"userId": $util.dynamodb.toDynamoDBJson($context.identity.sub),
"objectId": $util.dynamodb.toDynamoDBJson($context.args.objectId)
},
"attributeValues" : $util.dynamodb.toMapValuesJson($ctx.args)
}deleteObject mutation resolver:
{
"version" : "2017-02-28",
"operation" : "DeleteItem",
"key" : {
"userId": $util.dynamodb.toDynamoDBJson($ctx.identity.sub),
"objectId": $util.dynamodb.toDynamoDBJson($ctx.args.objectId)
}
}and getObject query to list metadata of objects owned by user:
{
"version": "2017-02-28",
"operation": "GetItem",
"key": {
"userId": $util.dynamodb.toDynamoDBJson($ctx.identity.sub),
"objectId": $util.dynamodb.toDynamoDBJson($ctx.args.objectId)
}
}for a url attribute of the Object type we define Lambda function resolver as below:
#set ($url = "https://cdn.dlozitskiy.online/$context.identity.sub/$context.source.objectId")
{
"version" : "2017-02-28",
"operation": "Invoke",
"payload": {"url": "$url"}
}This resolver logic will allow user to get signed URLs only for a content owned by the user.
AppSync CloudFront distribution
AppSync generates a random URL for GraphQL API which is not convenient when you building an App: every time you re-building the API you get a new URL and Front-end code will have to be changed. Therefore it is convenient to have a CloudFront distribution in front of AppSync, firstly because it allows us to use custom SSL certificate and secondly — Front-end doesn’t need to care about actual AppSync URL as it always points to CloudFront distribution URL. Let’s create another CloudFront distribution with custom SSL certificate for AppSync GraphQL API:
Create a Route 53 record that is pointing to the ClourFront distribution domain name:
Testing
To be able to test our API we need to create a user in a User Pool first. Once we have a user we will be able to get OpenID tokens from Cognito to call our API with. Let’s create a test user in our Cognito User Pool using AWS CLI, or you can do the same using AWS Congito Management Console:
$ aws cognito-idp admin-create-user --user-pool-id us-east-1_QZ3Aa0LBe --username tester --temporary-password 'password12345' --user-attributes Name=email,Value=testeer@tester.com Name=email_verified,Value=True
{
"User": {
"Username": "tester",
"Enabled": true,
"UserStatus": "FORCE_CHANGE_PASSWORD",
"UserCreateDate": 1542340635.642,
"UserLastModifiedDate": 1542340635.642,
"Attributes": [
{
"Name": "sub",
"Value": "e1f97eda-901e-4980-bbe2-015156ff7b71"
},
{
"Name": "email_verified",
"Value": "True"
},
{
"Name": "email",
"Value": "tester@tester.com"
}
]
}
}Note the “sub” of the user — this is what we will be using to create a test folder for this user in S3.
Once you created the user it will have a “FORCE_CHANGE_PASSWORD” as a status. We will need to initiate authentication process for the user to be able to change password. You can use Cognito Hosted Web UI (check out “Tips” section for that) or do the same using AWS CLI.
Let’s change user’s password by initiating authentication and responding to an authentication challenge:
$ aws cognito-idp admin-initiate-auth --user-pool-id us-east-1_QZ3Aa0LBe --client-id 5s6jmc25o0vm3vui34r9vh580j --auth-flow ADMIN_NO_SRP_AUTH --auth-parameters USERNAME=tester,PASSWORD='password12345'Respond to the authentication challenge with the new password:
$ aws cognito-idp admin-respond-to-auth-challenge --user-pool-id us-east-1_QZ3Aa0LBe --client-id 5s6jmc25o0vm3vui34r9vh580j --challenge-name NEW_PASSWORD_REQUIRED --challenge-responses NEW_PASSWORD='...',USERNAME=tester --session="..."We are going to use Insomnia Client for GraphQL to test our API while we don’t have a Front-end in place.
Let’s test a putObject mutation first: an Access Token received in previous step should be passed to AppSync as an Authorization header within Insomnia client:
Sending the request will create a new entry:
Let’s test the query of metadata for the test object:
Now let’s create a test folder under S3 bucket with user’s Cognito “sub” attribute as a name and place a report.pdf there:
Now let’s run the query to get a signed content URL:
If everything was setup correctly pasting this URL into browser or running CURL against it should start download of protected content:
Tips
While testing you might find useful to use Congito hosted Web UI for generating new identity and access token for Insomnia Client, the request will look like below:
https://appsync-api-access-user-pool.auth.us-east-1.amazoncognito.com/login?response_type=token&client_id=5s6jmc25o0vm3vui34r9vh580j&redirect_uri=https://google.comOnce you get authenticated the id and access token will be passed as a parameters in a callback URL similar to below, instead of google you can use any URL, even non-existing one
Next
In a next part we will look into building front-end part of our web application so stay tuned!
