Course Project

Objective

The goal of the project is to gain hands-on experience in building and deploying a scalable web service on the Internet. Using the latest web technologies while learning how to tackle the scalability and fault-tolerance concerns. This is a “learn by doing” course: the course project will form the primary focus of the course with the lectures and discussion of research papers providing background material. Each project will be conducted in an agile team where students will build their own scalable, redundant web site using fundamental web technologies and the Ruby on Rails framework.

Getting Started

Project Sprint Schedule

All sprints end and begin with each week’s lab session.

Sprint 0: September 28, 2017 – October 5, 2017

Sprint 1: October 5, 2017 – October 12, 2017

  • Complete Chapters 2 through 6 in the Ruby on Rails Tutorial.
  • Determine your team’s project, get approved by instructor.
  • Learn git.
  • Create and push the base rails application to your team’s github repository.

Sprint 2: October 12, 2017 – October 19, 2017

  • Complete chapters 7 through 10 in the Ruby on Rails Tutorial.
  • Learn TDD: get Travis CI working with your github repository.
  • Start writing stories for your project in Github projects.
  • Decide on a sprint commitment.
  • Learn pair programming through pairing up on the first few stories.

Sprint 3: October 19, 2017 – October 26, 2017

  • Complete chapters 13 and 14 in the Ruby on Rails Tutorial.
  • Conduct a retrospective on how the last sprint went and how you can improve.
  • Decide on a sprint commitment.
  • Implement stories from the current sprint.

Sprint 4: October 26, 2017 – November 2, 2017

  • Conduct a retrospective on how the last sprint went and how you can improve.
  • Decide on a sprint commitment.
  • Implement stories from the current sprint.
  • Learn EC2 and Amazon Web Console by deploying your initial application on Amazon EC2.

Sprint 5: November 2, 2017 – November 9, 2017

  • Conduct a retrospective on how the last sprint went and how you can improve.
  • Decide on a sprint commitment.
  • Implement stories from the current sprint. Your application should be mostly feature complete at this time and subsequent work should focus on polish.
  • Define the “critical paths” through your application (the set of pages that a common user will go through).

Sprint 6: November 9, 2017 – November 16, 2017

  • Conduct a retrospective on how the last sprint went and how you can improve.
  • Decide on a sprint commitment.
  • Implement stories from the current sprint. Your application should be feature complete at this time and subsequent work should focus on polish.
  • Deploy a tsung instance on EC2.
  • Write an initial tsung xml file to load test a simple action on your application.
  • Write a tsung xml file to exercise this critical path.
  • Load test your application with tsung running on a m3.medium instance.
  • Produce an initial set of graphs with your application’s performance.
  • Test (and document) the effects of vertical scaling on your application with the critical path xml file.
  • Deploy and test your application on a variety of load-balanced configurations, with the medium–large dataset.
  • Test using Tsung, measure, and document.

Sprint 7: November 16, 2017 – November 23, 2017

  • Conduct a retrospective on how the last sprint went and how you can improve.
  • Decide on a sprint commitment.
  • Implement stories from the current sprint. Only polish is appropriate at this point.
  • By the end of this sprint your project should be feature complete.
  • Create a medium–large dataset (~10,000+ records) using database seeds.
  • Test (and document) the effects of horizontal scaling on your application with the critical path xml file.

Sprint 8: November 23, 2017 – November 30, 2017 (No lab on the 23rd)

  • Conduct a retrospective on how the last sprint went and how you can improve.
  • Create a large dataset (more than 100,000 records).
  • Inspect and optimize the way your application is interacting with the database. If necessary, write custom sql to optimize.
  • Implement server-side caching (if you haven’t already)
  • Deploy your caching optimizations on a simple load-balanced configuration, with the large dataset, both with and without memcache.
  • Continue testing using Tsung, measure, and document.
  • How many users can you handle?

Sprint 9: November 30, 2017 – December 7, 2017

  • Conduct a retrospective on how the last sprint went and how you can improve.
  • If appropriate, create as large of a dataset as possible. How big can it be?
  • Continue testing using Tsung, measure, and document.
  • How many users can you handle?
  • Complete the project write-up (due Tuesday, December 5).
  • Prepare final presentation (present Thursday, December 7).