Hosting on Amazon Web Services (AWS)

A few weeks back I posted “So What’s the Business?” and briefly discussed hosting my site in the cloud as opposed to trying to build my own infrastructure, and I touched on the serverless option as well, but I didn’t go into depth at all. Now I’d like to do that a bit regarding hosting.

There are thousands of options for hosting a site in the cloud. Just do a web search on “web hosting” and there are thousands in the United States alone. There are the big companies, smaller ones, and probably even Trump’s 400-pound guy has a server available to host your site.

But seriously, there are a lot of options to choose from. I am going to be hosting my business on one of these options, so it is vital that it has these qualities:

  • Dependable
  • Reliable
  • Can handle large fluctuations in traffic
  • Excellent support available
  • Great documentation available
  • Good community of users
  • A good track record
  • Mature in its offerings
  • Ease of use
  • Affordable
  • Familiar
  • Has serverless capabilities

I’m sure there are more but this is what comes to mind right now. If I am going to have my business completely dependent upon another business to make sure it is always available and operating properly it had better be damn good.

This list, especially the last one regarding serverless, leads me to the big company options: AWS, Google Cloud or Azure. Having gone to the ServerlessConf I learned a lot about what is available.  AWS is the most mature, though the others are catching up. I got to work with AWS during my workshops at the conferences, I have worked with AWS at work and I have been studying it via LinuxAcademy for which my current job is providing an account for training. I feel comfortable with AWS and am impressed with it. Also it meets all of my criteria.

icon-cloud-aws

If you are interested to learn more, this is a good chart of the three with links to their various offerings (I got the table here).

Microsoft Azure Amazon Web Services (AWS) Google Compute
Available Regions Azure Regions AWS Global Infrastructure Google Compute Regions and Zones
Compute Services Virtual Machines (VMs) Elastic Compute Cloud (EC2) Compute Engine
App Hosting Cloud Services
Azure Websites and Apps
Azure Batch
Azure Scheduler
Logic Apps
Amazon Elastic Beanstalk Google App Engine
Serverless Computing Azure Functions AWS Lambda Google Cloud Functions
 ALM & Code Editor Azure Visual Studio Online AWS CodeDeploy None
Container Support Docker Virtual Machine Extension (how to) EC2 Container Service

Container Engine
Container Registry
Scaling Options Azure Autoscale (how to) Auto Scaling Autoscaler
Analytics/Hadoop Options HDInsight (Hadoop) Elastic MapReduce (EMR) Google Cloud Dataproc
Government Services Azure Government AWS GovCloud None
App/Desktop Services Azure RemoteApp Amazon WorkSpaces
Amazon AppStream
None
Object Storage Azure Storage (Blobs, Tables, Queues, Files) Amazon Simple Storage (S3) Cloud Storage
Block Storage Azure Blob Storage (how to) Amazon Elastic Block Storage (EBS) Persistent Disk
Hybrid Cloud Storage StorSimple AWS Storage Gateway None
Backup Options Azure Backup Amazon Glacier Google Cloud Storage
Disaster Recovery Planning Azure Site Recovery None None
Content Delivery Network (CDN ) Azure CDN Amazon CloudFront Cloud CDN
Database Options Azure SQL Database Amazon Relational Database Service (RDS)
Amazon Redshift
Cloud SQL
Cloud Spanner
NoSQL Database Options Azure DocumentDB Amazon Dynamo DB Cloud Bigtable
Cloud Datastore
Caching Azure Managed Cache (Redis Cache) Amazon Elastic Cache None
Data Orchestration Azure Data Factory AWS Data Pipeline BigQuery
Cloud Dataflow
Networking Options Azure Virtual Network Amazon VPC Cloud Virtual Network
Azure ExpressRoute AWS Direct Connect Cloud Interconnect
Azure Traffic Manager Amazon Route 53 Cloud DNS
Load Balancing Load Balancing for Azure (how to) Elastic Load Balancing Cloud Load Balancing
Administration & Security Azure Active Directory AWS Directory Service
AWS Identity and Access Management (IAM)
Cloud Identity & Access Management (IAM)
Multi-Factor Authentication Azure Multi-Factor Authentication AWS Multi-Factor Authentication Cloud Identity-Aware Proxy (IAP) (Beta)
Security Key Enforcement
Monitoring Azure Operational Insights Amazon CloudTrail Cloud Console
Azure Application Insights Amazon CloudWatch Stackdriver Monitoring
Stackdriver Logging
Queueing Azure Service Bus
Azure Event Hubs
Amazon Simple Queue Service (SQS) Cloud Pub/Sub
Notifications Azure Notification Hubs Amazon Simple Notification Service (SNS) None
Secure Credentials Azure Key Vault (Preview) AWS Key Management Service Cloud Key Management Service
Compliance Azure Trust Center AWS CloudHSM Google Cloud Platform Security
Management Services & Options Azure Resource Manager Amazon CloudFormation Cloud Deployment Manager
API Management Azure API Management Amazon API Gateway Cloud Endpoints
Automation Azure Automation AWS OpsWorks
AWS Config
Compute Engine Management with Puppet, Chef, Salt, and Ansible
Automated Image Builds with Jenkins, Packer, and Kubernetes
Search Service Azure Search Amazon CloudSearch None
Analytics Azure Stream Analytics Amazon Kinesis Cloud Dataflow
Cloud Dataprep (Beta)
Email Services None Amazon Simple Email Services (SES) None
Media Services Azure Media Services Amazon Elastic Transcoder Cloud Video Intelligence API
Machine Learning Azure Machine Learning (Preview) Amazon Machine Learning Cloud Machine Learning Engine
Workflow Azure BizTalk Services Amazon Simple Workflow (SWF) None

Free Tiers

All three of them have free options that allow you to do development and test without paying anything or at least very little. Here’s an article that compares the three free tiers.

I however am interested in AWS specifically and this is their free tier. Some last for just 12 months and some are always free.

Where I Am Now

So far I have been using AWS mostly through my own study. I am currently working through this book and will share what I learn in future posts. As I go through the book I am going to design my site so that once I get through the book I can start implementing it. I hope to have a basic version of the site complete by the end of August.

 

Symphonia’s Serverless Insights for June 15, 2017

“Welcome to the latest issue of Symphonia’s Serverless Insights newsletter. Our mission is to bring the benefits of Serverless to both our clients and the industry at large, and we’re excited to share our latest thoughts with you…..”

There is a lot of useful information in these newsletters and I will share them as they come out, or you can subscribe for yourself here.

If you know of any other good resources please let me know and I’ll consider adding them as well!

OSCON Austin 2017 – Keynotes & Sessions

Continuing my previous post from Tuesday, the last two days of the conference were keynote speeches and talk sessions. I am not going to discuss every session I attended as there were many, I’ll just hit the ones that were the most interesting for me.

To give you an idea of how many sessions there were, look at the image below. Every hour there were 14 different sessions to choose from! And of course as you would know it, the ones I really wanted to attend most were all at the same times and my current employer didn’t pony up the big bucks to get me access to videos of all of the sessions, so I was forced to choose.

Wed Sched

14 Different Choices Every Hour

Keynotes

So the beginning of days 3 and 4 were keynote addresses, 6 on Wednesday and 4 more on Thursday. Unfortunately O’Reilly only posted excerpts for most of the keynotes on their YouTube channel. To see the whole videos you need a Safari subscription. I’ll include the ones with the full videos below.

Christopher Aedo from IBM discussed open infrastructures and defaced numerous classic paintings.

Ying Xiong, the Chief Architect of the cloud platform at Huawei Technologies, spoke about the open source ecosystem. This was very interesting to get a different point of view from China.

Finally I hate to include this one but I will anyway since the whole video is there and there is some useful information in it. Alvand Salehi, a Senior Technology Advisor at The White House, discussed open source from the U.S. Government, even some code from the Pentagon.

So that aspect was interesting. It was just unfortunate that I felt like I was buying a used car.

wagon-queen-family-truckster-national-lampoons-vacation

There’s nothing like the Family Truckster

Sessions

Over the 2 days I attended 12 sessions, and there were 3 that stood out:

“From REST to GraphQL: Why a query language is perfect for writing APIs”

I could really beat myself with a wet virtual network cable. I got to this one late. It was the first one after the keynotes and I spent some time in the Expo Hall and lost track of time. The room was packed except for some seats up front (I don’t see why 50 people or more were just standing up in the back when there were seats up front). It was hard to see the screen but it was well worth it.

IMG_0390

The session was taught by David Celis and Garen Torikian of GitHub. They tag teamed the story of a fictional company called Welp and their need to develop a REST API. After documenting the inherent problems of REST they decided that they needed to use GraphQL instead! What followed was a really good live demo along with tons of great details.

I know that when I get to the point of building out my API I will be spending a ton of time going through their slides and examples. Really great stuff. Thanks guys!!

Slides (with notes!): From REST to GraphQL

UPDATE: They provided the video to their session! Now I can see what I missed and everything else as well!

“The serverless revolution for JavaScript developers”

This presentation by Pam Selle (and her blog), a software engineering lead at IOPipe, didn’t have much new for me that I hadn’t already heard at ServerlessConf or earlier at OSCON, but it was by far the best talk to describe and detail what serverless is and what the different options are if you want to go down this road.

If you aren’t really sure what this thing called serverless really is, go through her slides. I wish her presentation video were available as well because that makes it so much more clear. Definitely a good read and a good resource. Thanks!

Slides: The serverless revolution for JavaScript developers Presentation

UPDATE: Pam provided the video to her session!! Thank you!!

“Graph databases will change your freakin’ life”

This was an excellent talk on Graph databases bu Ed Finkler, the CTO at Graph Story. After attending that workshop by William Lyon two days earlier my interest was intense and this talk didn’t disappoint. One thing that does disappoint is that his slides were not posted afterwards 🙁 (By the way, most of the slides to the talks can be found here).

UPDATE: He provided the slides!! Also, I found a video of him making a similar (or possibly the same) presentation previously and here is the video (embedding is turned off).

His talk covered a lot of the same material that the workshop did, but that only served to deepen my understanding of it. Plus seeing how someone else used Cypher and Neo4J really helped my understanding.

Definitely check out his non-profit organization called OSMI which stands for Open Sourcing Mental Illness. Ed is doing some excellent work. Give what you can to help. I did!

Also, he has provided an example repo in Github called 2016 OSMI Survey Graph. Check it out! I plan to.

OSCON Austin 2017 – Workshops

Recently I posted about my experience attending the Serverless Conference in Austin at the end of April. Two weeks later I attend OSCON in Austin as well. OSCON stands for Open Source CONvention and is put on by O’Reilly.

OSCON

Got there early, waiting for the keynotes to begin.

The convention was held over 4 days. The first two days were workshops, 2 a day. The last two days were a combination of keynotes and hour long talks. I really would have preferred that the workshops came at the end instead of the beginning, but this was just as good I guess.

Given the vast amount of information I received over these 4 days I’ll split this into two posts, this one will be on the 2 days of workshops, followed by one post on the 2 days of keynotes and talks.

Day 1

Workshop 1:Building, deploying, and running a scalable and extensible serverless application using AWS

The workshop was taught by the two founders of Symphonia, an expert Serverless and Cloud technology consultancy based in New York City (Make sure to get their free e-book “What is Serverless?” available on their website now). John Chapin and Mike Roberts tag teamed the workshop to great success, leading us through a lesson on what exactly serverless is, talking about AWS, Lambda and Java, and then finally taking us through a tutorial.

Symphonia

They planned their course well as we learned a great deal without feeling overwhelmed (at least I didn’t). Having already attended workshops and sessions at the Serverless conference two weeks prior probably helped me a great deal in understanding.

One way this workshop was different though is that the code was all in Java and used Maven. The workshop I attended at the Serverless Conference used NodeJS and npm. I spent 8 years as a Java developer, I even taught Java at the college level a couple of semesters way back right after I finished Grad School in 2000. It all felt very familiar which was wonderful.

I had a chance to talk with John Chapin briefly at both conferences. I didn’t get to talk with Mike Roberts but I did get to see him speak at the Serverless Conference. They are both very knowledgeable and easy to talk with.

I look forward to contacting them for advice and guidance as my business develops. Thanks for a great workshop!

Reference Material:

Workshop 2: “Building a deployment pipeline with Jenkins 2.0”

This workshop was on a topic I felt I should know or at least be exposed to more, mainly for my current full-time job that pays the bills, and possibly for myself down the line. The workshop was taught by Brent Laster who was obviously well versed on the subject.

Jenkins

Sitting a little too close!

Unlike the previous workshop, this felt like a double-barreled fire hose. We went through topics and exercises quickly and I was doing what I could to keep up. When I get into that mode I am happy when I complete the tasks and keep up, but I really don’t learn anything along the way.

After a bit he said that he designed this course to have as much information as possible so we can learn it afterwards on our own time and have a lot of reference material. While that is nice to have after the workshop, during it I felt I was getting very little out of it. I would have rather gone at a slower pace during the class and had more time to digest so I would retain more. Now I just have a bad taste in my mouth about it and will only go back to the reference material when I need to.

He did mention this book above, Continuous Delivery, as the main source of his material for the workshop. I provide a link here as a first attempt at monetizing the site a bit. 🙂 Every penny helps!

Reference Materials:

Day 2

Workshop 3: “Power Git: Rerere, bisect, subtrees, filter branch, worktrees, submodules, and more”

So my first workshop on the second day was a lot like my last one from the day before. Something I felt I should know more about though I wasn’t that excited about it. Git however is something I do use every day and maybe I could get more out of it if I get to know it better?

Git

I picked a better seat for this one.

The workshop was taught by the same person who taught the Jenkins workshop, Brent Laster, so it was a very similar format. I’m more familiar with Git so it seemed more like a single-barreled fire hose than a double. This was also a bit more straight forward as well.

He mentioned a book where he got a lot of material for the workshop, and this time it is a book he wrote himself!

Work through the labs if you want to learn more about some higher level functions of git. I don’t ever need them with the work I do, but it’s nice to be a little familiar with them and to be able to quickly look up what I need.

Reference Materials:

Workshop 4: Building a real-time recommendation engine with Neo4j

My final workshop was my biggest surprise. I knew what I was getting with the first one and was excited about it, the next two were all about acquiring necessary knowledge, but the last was just something that I thought might be interesting.

I honestly hadn’t thought about it much going into it. I got what I was looking for with the first workshop and felt numb after the next two, so when I walked in to this one I had a blank slate. No expectations.

Node4j

Sat way too far away for this one!

The workshop was taught by William Lyon from Neo4J. We first got into what the Graph Database Model is and how it differs from relational databases. The idea of nodes and relationships instead of tables and rows really fascinated me. The more we learned the more powerful I realized this format was.

Then we started to get into the query language for Neo4J called Cypher. This took a while to get used to. I have spent a lot of time writing SQL queries over the years, but Cypher takes a shift in thinking. By the end I felt I was really starting to catch on.

He pulled data from the Meetup.com site for the Austin area and we were able to query it. I’m a member of a few (though like an idiot I never go) so I was able to query myself.

We did all of this querying in the Neo4J sandbox which uses the Neo4J browser and it has an amazing interface. I really fell in love with using it. I want to find a use for it just so I can use it (actually I am sure I’ll have a use case for graph databases soon enough). I really look forward to going back through all of this and exploring further when I get to that point in my app development. Thanks William for a great workshop!

Reference Material:

OSCON Austin 2017 Part 2 Coming Soon….

ServerlessConf Austin 2017

Over the past month and a half I have attended two IT conferences here in my hometown of Austin: ServerlessConf and OSCON. I had an idea that these might be transformative, but I had no idea to what extent that would become true! I’ll talk about OSCON in a later post, but now I’ll give you my take on the ServerlessConf.

ServerlessConf – Workshop

ServerlessConf.png

This was a 3-day conference, but going into it I was excited just about the first day. That day would be an all-day workshop called “Build Your Own Serverless Video Sharing Website – with Lambda, API Gateway and Firebase”. Here is the description of the workshop:

“Learn how to build your very own video sharing website, like YouTube, using serverless technologies. During the workshop you will build an entire end to end system (along with an HTML5 website) that’s powered by AWS Lambda and API Gateway, S3, Elastic Transcoder, Firebase, Auth0 and much more. As you are guided through the workshops you will also learn about useful patterns and architectures such as “compute as backend” and “compute as glue”. This workshop is suitable for beginners and experts alike (and if you are an expert we’ll be sure to challenge you just a little bit more)”.

What was so exciting for me was this was the beginning of what I want to do with MY site, MY business! Not only was I going to learn serverless technologies, I was going to get an example of what I was planning to build!

The workshop lived up to my expectations and more. The instructors Sam Kroonenburg and Daniel Parker, both of A Cloud Guru, prepared an excellent course. They even provided each of us the course in book form, with easy to follow step-by-step instructions. You can still find the course here on github.

Given we only had one day to get through it all, the code we needed was already written for us. We modified a few things here and there, but mostly our work happened in AWS. Unlike other workshops that give you too much to accomplish in the time provided, I was able to get all of this done and then spend time going over it and letting it all soak in. This was a firehose of knowledge I could handle.

tumblr_inline_oqxf7b0DfB1qgbyen_400

Peter Sbarski, also of A Cloud Guru, and who was there assisting this workshop as well as others, has written a book using this workshop as a starting point and then adding a LOT to it. It is called Serverless Architectures on AWS. I am currently working through it as often as I can. It is well written and thorough. It is what I need to get my business off the ground and I am so grateful to have it! If you have Safari it is also now available there.

ServerlessConf – Speakers

The next two days of the conference I initially thought would be a let down, as I was so geared toward that workshop, but I was pleasantly surprised by the depth of knowledge I gained from the speakers.

All of the speakers were recorded and you can find them here on YouTube. If you are interested in these technologies then these are definitely worth watching. There was not only AWS, but Google Cloud, Microsoft Azure and IBM’s OpenWhisk as well.

tumblr_inline_oqxgxdsgef1qgbyen_500

One talk that really stands out to me and that I’ll be referring to frequently as I develop my application is the one on Serverless Security by Guy Podjarny (here is his blog). This is such an important topic and he covered it all so well. I wish he would write a book on this subject!

Thanks to all for such a great conference, especially to those from A Cloud Guru who started this. You gave me so much and really jump started my business development. I’ll forever be in your debt!

So What’s the Business?

So far I haven’t given much of an indication as to what the business is going to be. That is intentional as I plan to build up to that as I build the business. But I will give you some hints as to what my idea is and I will definitely detail how I plan to build it.

iStock_000069453941_Large

The business will initially be strictly online and something that anyone can find something to enjoy, learn from, and even profit from. It’ll also give you a way to give to your favorite charities or other non-profit organizations in a very creative way that will entice others to give as well.

Given the business will be online, I obviously have a website to build! Good thing I’ve been involved in that kind of work for quite a while now. But the tools I plan to use are very new, some are just being made available and not fully mature. Why would I risk a new business on untested technology?

As the name of this blog reveals, I am bootstrapping this new business. What exactly does that mean? I am not getting any outside funding, its all what I can put into it myself and what I can earn from it that will keep it going.

As a result, I need to keep expenses as minimal as possible. With a website there are definite costs, such as hosting, scaling, storage, support, etc. These costs can vary quite a bit, so how to minimize?

Hosting on Home Servers Option

I could do what used to be done, buy or build my own server and host it at home. I’d have total control over it but then I’d also be the sole support of it. Also what happens as the business grows (hopefully)? Do I buy more servers to allow it to scale, more storage as the needs grow? Where does that end?

Maybe this would be something I would want to do eventually when the business can afford to operate its own infrastructure with full support and personnel, but that is a long way off. There has to be something else.

Hosting on the Cloud Option

How about the cloud? This is a mature technology now and I have had multiple years of experience working in it. There is no equipment to buy or maintain, just pay for the virtual machines and services that you use. Someone else makes sure it is always up and running and is there to help when you need it, plus you can automate it to quickly scale as the load fluctuates. I wouldn’t want my website going down right when its getting busy or running at full capacity when no one is on it.

This sounds much better but there is a small problem. When you create virtual servers in the cloud to host your application, you pay for it 24 x 7, as long as it is running and you ALWAYS want it running. As your business grows you’ll need more servers, and that cost will increase. Is there any way to get these costs down?

The Serverless Option

severless

What is serverless? Does this mean hosting computers without a server? NO! It does not. You still have to have a server, hopefully multiples of them as your business is booming. What it means is that the details of the server are abstracted away from you. You no longer have to manage it, you don’t have to handle the scaling, you just have to use it (and of course monitor it).

And you only pay for exactly what you use!! If no one is visiting your site (hopefully that’ll never happen) then you don’t pay for it to be there. It’ll still be there when someone visits.

But to get the full benefit of this technology you have to design your site to take advantage of it. The old monolith apps of days gone by (well, today still) won’t give you any benefit. It’s all about microservices baby!

I’m going to delve into each of these topics over the coming weeks, and also talk about a couple of conferences I attended to learn about all of this: ServerlessConf and OSCON.

For now though I have to finish packing. I have to head to the airport in an hour! Have a wonderful Memorial Day weekend everyone!