Screaming in the Cloud

Episode 52: AWS as a Career Finding Clarity in the Ever-changing Job Market of the Cloud

Corey Quinn — Wed, 20 Mar 2019 10:43:00 +0000

The job market in the AWS world is complex and often confusing to both employers and employees. Wouldn’t it be great to have over 43,000 data points to draw a larger picture of the market and where you fall in line?

Today, we are talking to Kate Powers who walks us through the AWS Salary Survey from Jefferson Frank and discusses some interesting insights as well as real world examples of the findings.

Some of the highlights of the show include:

The AWS job market at large
Training Certificates: what’s their value
How much value is in a job title
Most desirable skills from employers
Gender representation in the industry
The discrepancy in compensation based on geography

Links:

Episode 51: Size of Cloud Bill: Not About Number of Customers, but Number of Engineers You've Hired

Corey Quinn — Wed, 06 Mar 2019 06:00:00 +0000

Years ago, if you wanted to launch an Internet company or Web application, you had to own necessary hardware. Now, the economics have changed drastically with the ease of Cloud computing. It’s still a new industry that people are trying to figure out, especially when it comes to cost and optimization.

Today, we’re talking to Dann Berg, a Cloud ops analyst at Datadog. He helps others understand and lower the cost of Cloud operations. Dann is a detective who is dedicated to figuring out why a company’s Cloud bill is so high.

Some of the highlights of the show include:

Companies struggle with field of Cloud economics; can be overwhelming because there’s so much to learn about products and implementation
Companies use the Cloud to grow quickly, which makes their Cloud costs grow quickly and more than expected
Only access to full list of every resource being used is the Cloud bill; there’s no comprehensive inventory service available
Companies need to offer visibility to Cloud bill; not everyone has access to understand how their actions impact the bill
Cost of Cloud bill is dependant on different factors, including new features, new users, and cost of goods sold (COGS)
Scale and manage bill by using a platform app or hiring a consultant/team
Understand pricing of AWS and learn best practices for cost controls early on
Don’t leave money on the table by focusing on engineering time - not best use of resources; focus on the smallest things that have the biggest impact
Cost is important, but don’t slow down those developing in the Cloud; open lines of communication to create culture to understand cost, value what’s measured

Links:

Episode 50: If You Lose Data, Your Company is Having a Very Bad Day

Corey Quinn — Wed, 27 Feb 2019 17:34:00 +0000

If you use MongoDB, then you may be feeling ecstatic right now. Why? Amazon Web Services (AWS) just released DocumentDB with MongoDB compatibility. Users who switch from MongoDB to DocumentDB can expect improved speed, scalability, and availability.

Today, we’re talking to Shawn Bice, vice president of non-relational databases at AWS, and Rahul Pathak, general manager of big data, data lakes, and blockchain at AWS . They share AWS’ overall database strategy and how to choose the best tool for what you want to build.

Some of the highlights of the show include:

Database Categories: Relational, key value, document, graph, in memory, ledger, and time series
AWS database strategy is to have the most popular and best APIs to sustain functionality, performance, and scale
Many database tools are available; pick based on use case and access pattern
Product recommendations feature highly connected data - who do you know who bought what and when?
Analytics Architecture: Use S3 as data lake, put in data via open-data format, and run multiple analyses using preferred tool at the same time on the same data
AWS offers Quantum Ledger Database (QLDB) and Managed Blockchain to address use case and need for blockchain
Authenticity of data is a concern with traditional databases; consider a database tool or service that does not allow data to be changed
Lake Formation lets customers set up, build, and secure data lakes in less time
DocumentDB: Made as simple as possible to improve customer experience
AWS Culture: Awareness and recognition that it takes many to conceive, build, launch, and grow a product - acknowledge every participant, including customers

Links:

Episode 49: Open Source Software: Slipping Beneath the Surface of Awareness

Corey Quinn — Wed, 20 Feb 2019 06:00:00 +0000

Does operating system (OS) choice even matter anymore to most people? Especially with the emergence of serverless and containers? Debian may not see its name up in lights much these days, but it’s still very much front, center, and relevant to what people are doing in Cloud environments.

Today, we’re talking to Elana Hashman, a Python packager and Debian developer. Everything inside a base operating system may not be interesting to end users, but such a collection of components is necessary to create a functioning Linux system.

Some of the highlights of the show include:

Alternative Linux operating systems, including Amazon Linux 2
Level of awareness about free software when choosing and distributing an OS
What is a Python packager? How do you become one?
Python is the new default language due to growth and adoption of its ecosystem
Packaging community off-putting to beginners; find someone who understands the system to guide you

Links:

Episode 48: Nobody Gets Rid of Anything, Including Data

Corey Quinn — Wed, 13 Feb 2019 06:00:00 +0000

Companies can find working in the Cloud quite complicated. However, it’s a lot easier than it used to be, especially when trying to comply with regulations. That’s because Cloud providers have evolved and now offer more out-of-the-box services that focus on regulation requirements and compliance.

Today, we’re talking to Elliot Murphy. He’s the founder of Kindly Ops, which provides consulting advice to companies dealing with regulated workloads in the Cloud.

Some of the highlights of the show include:

Technical controls are easier, but requirements are stricter
Risk Analysis: Putting locks on things to thinking about risks to customers
Building governance and controls; making data available and removable
Secondary Losses: Scrub services to make scope and magnitude of loss smaller
Computing became ubiquitous and affordable; people started collecting data to utilize later - nobody gets rid of anything
General Data Protection Regulation (GDPR) set of regulations apply to marketing technology stacks to manage systems
Empathy building exercise and security culture diagnostic help companies understand compliance obligations
Security Culture: Beliefs and assumptions that drive decisions and actions
Evolution of understanding with public Cloud’s security and availability
Raise the bar and shift mindset from pure prevention to early detection/ mitigation; follow FAIR (factor analysis of information risk)

Links:

Episode 47: Racing the Clouds

Corey Quinn — Wed, 06 Feb 2019 06:00:00 +0000

More and more enterprises and on-prem applications are moving to the Cloud. Therefore, flexibility, agility, time-to-market, and cost effectiveness need to be created to address a lack of visibility and control.

Today, we’re talking to Archana Kesavan, senior product marketing manager at ThousandEyes. The company offers a network intelligence platform that provides visibility to Internet-centric, SaaS, or Cloud-based enterprise environments. Our discussion focuses on ThousandEyes’ 2018 Public Cloud Performance Benchmark Report.

Some of the highlights of the show include:

Purpose of Report: Reveals network performance and architecture connectivity for Amazon Web Services (AWS), Google Cloud (GCP), and Microsoft Azure
Report gathered more than 160 million data points by leveraging ThousandEyes’ global fleet of agents that simulate users’ application traffic
Data collected during four-week period was ran through ThousandEyes’ global inference engine to identify trends and detect anomalies
Internet X factor when calibrating network performance of public Cloud providers; best-effort medium that has no predictability and is vulnerable to attacks
AWS’ performance predictability was lower than GCP Cloud and Azure leveraged their own backbones to move user traffic
Certain regions, such as Asia, were handled better by GCP and Azure than AWS
Customers should understand value of long-distance Internet latency when selecting a Cloud provider
Determine what the report’s data means for your business; conduct customized measurements for your environment

Links:

Episode 46: Don't Be Afraid of the Bold Ask

Corey Quinn — Wed, 30 Jan 2019 17:17:00 +0000

If you’re looking for older services at AWS, there really aren’t any. For example, Simple Storage Service (S3) has been with us since the beginning. It was the first publicly launched service that was quickly followed by Simple Queue Service (SQS). Still today, when it comes to these services, simplicity is key!

Today, we’re talking to Mai-Lan Tomsen Bukovec, vice president of S3 at AWS. Many people use S3 the same way that they have for years, such as for backups in the Cloud. However, others have taken S3 and ran with it to find a myriad of different use cases.

Some of the highlights of the show include:

Data: Where do I put it? What do I do with it?
S3 Select and Cross-Region Replication (CRR) make it easier and cheaper to use and manage data
Customer feedback drives AWS S3 price options and tiers
Using Glacier and S3 together for archive data storage; decisions and constraints that affect people’s use and storage of data
Feature requests should meet customers where they are, rather than having to invest in time and training
Different design patterns and best practices to use when building applications
Batch operations make it easier for customers to manage objects stored in S3
AWS considers compliance and retention when building features
Mentorship: Don’t be afraid of the bold ask

Links:

Episode 45: Everybody Needs Backup and Recovery

Corey Quinn — Wed, 23 Jan 2019 16:01:00 +0000

Do you have to deal with data protection? Do you usually mess it up? Some people think data protection architecture is broken and requires too many dependencies. By the time a business needs to backup a lot of data, it’s a complex problem to go back in time to retrofit a backup solution for an existing infrastructure.

Fortunately, Rubrik found a way to streamline data protection components. Today, we’re talking to Chris Wahl and Ken Hui of Rubrik.

Some of the highlights of the show include:

Transform backup and recovery to send data to a public Cloud and convert it to native format
Add value and expand what can be done with data - rather than let it sit idle
Easy way for customers to start putting data into the Cloud is to replace their tape environment; people hate tape infrastructure more than their backups
Necessity to backup virtual machines (VMs) probably won’t go away because of challenges; Clouds and computers break
Customers leaving the data center and exploring the Cloud to improve operations, utilize automation
Business requirements for data to have a level of durability and availability
People vs. Technology: Which is the bottleneck when it comes to backups?
Words of Wisdom: Establish an end goal and workflow/pathway to get there

Links:

Episode 44: Disagree In Commits Console Recorder for AWS

Corey Quinn — Wed, 09 Jan 2019 06:00:00 +0000

Do you have some spare time? Can you figure out an easier way to do something? Then, why not build some software?!

Today, we’re talking to Ian Mckay of Kablamo, an Amazon Web Services (AWS) consultancy. He is the author of Console Recorder, which is a browser extension that records your actions in the Management Console to convert them into SDK code and infrastructure as code templates.

Some of the highlights of the show include:

Timeline to build Console Recorder
Infrastructure as Code: How to code repeatedly without starting over and take ownership of what you built by hand
AWS vs. Individual Achievements: People asked AWS for years to create something to record console click-throughs that Ian did in his spare time
Console Recorder support for any browser that exports Web extensions
Sharp edges of what’s expected of Console Recorder to speed up development
Management Console’s unreadable responses require reverse engineering
Console Recorder: Recommended use cases and areas
How to alleviate security concerns with Console Recorder
Changes to Management Console that may break things
Ian’s past, present, and future projects and products
Words of Wisdom: If you don’t like something, just fix it yourself

Links:

Episode 43: Here’s a Document on How to Best Deal with My Foibles

Corey Quinn — Wed, 02 Jan 2019 06:00:00 +0000

A Manager README is a document designed to establish clarity between a manager and those who report to them. These documents are especially useful for onboarding content. For example, if you have someone new starting on your team, there's so many things you need to share with them - pieces of advice and guidance that help them to make the best decision about what to do in specific situations. A Manager README sets some expectations in advance to make things easier and reduce friction and anxiety for team members.

Today, we’re talking to Matt Newkirk, who manages Etsy’s localization and translation group. He explains that even if your company has an intensive onboarding program and review process, some things are still left out. A Manager README is a helpful and proactive piece of content that prompts conversations about how people perceive things.

Some of the highlights of the show include:

Avoid writing READMEs that are extremely self-centered/arrogant
READMEs clarify what to do until a relationship is established between the manager and their employee
Get feedback early on to make sure that what you include in the document is helpful; it should reflect reality and be discussed
Share README with your manager to make sure you’re both on the same page about team philosophies and expectations
README is a living document that needs to be updated occasionally because things change
README adds context; it’s not designed to make employee feel like they’re back in school and panicking because they’re not prepared
Manager README - Not Matt’s best selection of terminology
Who’s the best boss you ever had? Why? They can be a force that shapes your life and career from the right perspective
Philosophy of Management: Don’t do what terrible managers have done; be transparent about strategic reasons for priorities changing

Links:

Episode 42: SCREAMING WITH CHAOSSEARCH: A reInvent reTrospective

Corey Quinn — Wed, 26 Dec 2018 06:00:00 +0000

Would you like access to unlimited retention of your data within your Amazon S3, which costs far less than online storage on disc? Well, the next time you’re at re:Invent, visit CHAOSSEARCH’s booth.

Today, we’re talking to Pete Cheslock, vice president of products at CHAOSSEARCH and former vice president of operations at Threat Stack. CHAOSSEARCH helps people get access to their login event data using Amazon S3.

Some of the highlights of the show include:

re:Invent - Year of the Pin: People go nuts for conference swag and were collecting pins as if they were gold
Scan Your Badge and Drip Emails: Annoying and passive-aggressive marketing trends meant to be spontaneous and interesting
Need a job? Corey’s looking to hire a “Quinntern” to use a tag email address to gather conference swag at the next re:invent; if interested, contact him
Corey and Pete’s Swag Rules: Something you want or can use, continues to be valuable, no sizes, no socks
Densify Drama: Conference flyer to generate leads failed, created complaints
Track and analyze data, but don’t use it to invade privacy or become creepy
Las Vegas: Right place for conferences, such as re:Invent?
Rather than focusing on going to conference sessions, make meeting and talking to people doing interesting things your priority
Midnight Madness Event: Only place Corey could do stand-up Cloud comedy
re:Invent 2019: Plan appropriately, identify what you want to get out of it, register ASAP to get a nearby hotel, and schedule meetings with AWS staff

Links:

Episode 41: Open Source is Not a Business Model

Corey Quinn — Wed, 19 Dec 2018 06:00:00 +0000

Have you ever had high expectations about a new software product? Did you think it was going to be spectacular? Instead, did it become less about solving a problem for you and more about reaching a bunch of billable consultants? The dynamics of open source communities and the Cloud platform can make or break software products.

Today, we’re talking to Andrew Clay Shafer, who was a notable voice during the days of OpenStack. He had high hopes for OpenStack, which was an effort to bring a democratized solution of Cloud computing to anyone’s data center. He describes the importance of understanding the challenges associated with open source projects in order for them to be successful.

Some of the highlights of the show include:

Open source is not a business model; capture value for customers, or they’ll go with a different solution
Openness/Closure: Every open source project has its own community dynamics
Losing sight of level of expertise for profitability and easy path to useage
Whether to become a product or service company - difficult to be both effectively or go from being one to the other; build partner relationship, focus, and say “no”
Lack of awareness about AWS Outposts admitting public Cloud is no longer a viable business model
Amazon relentlessly focuses on what its customers want and tries to keep promises about what it can and can’t do
Cloud Native: Not where you run, but how you run; confining variables
Self-fulfilling prophecy to under deliver when you make the bad decision to under source IT across the board
Cloud Native, DevOps, SRE: Buzzwords that equal one thing and work together
Dilemma of not building everything and buying some things, but you can’t buy everything; humans like to shop and go with the easiest option

Links:

Episode 40: Wave of Innovation Breaking Ahead of the Bow of the Ship that is Amazon

Corey Quinn — Wed, 12 Dec 2018 06:00:00 +0000

You can't make money selling to developers! The bottleneck of getting business requirements and creating business value used to mean waiting for the next waterfall release. That’s not the case anymore in the venture community. There’s programmatic access to infrastructure and DevOps/agile developments that offer super-fast cycle times. Now, the bottleneck is about how fast your developers can move and how much they can get done.

Today, we’re talking to Joseph Ruscio, general partner at Heavybit Industries, which is an accelerator for seed-stage companies and focuses on developer-first products. Tools and products that get you more leverage out of your developers are incredibly valuable.

Some of the highlights of the show include:

Measuring maturity of startups’ engineering teams by looking at SaaS list - what products they have in place and how many are using out-of-house vendors
Customers don’t care how curated or artisan a piece of your stack is, they only care that it works
Not all claims (scales infinitely or never fails) are true when it comes to products on the market, so people are skeptical
Heavybit focuses on helping businesses build a bottoms-up, grassroots community around its products and a disciplined inside/direct sales motion
Build vs. Buy: Whatever people try to do themselves is a costly, pale imitation of something they can buy
Advice for New Entrepreneurs: Never compete with AWS on hosting compute because it will obliterate and Amazon is great at plumbing, terrible at painting
AWS’s version of your product won't be as sophisticated; continually work on it to deliver a more seamless product and customer success experience
Measure downtime/outages in terms of dollars by using monitoring tools that deliver more holistic, integrated, comprehensive experience than CloudWatch
Starting a company is easier; even if you're the 800-pound gorilla in the category you created, keep innovating and building or Amazon’s coming after you
Azure, unlike GCP, has ability to meet customers where they are, rather than telling them where they should be
Understand the problem your customer is trying to solve and understand how far out of their current comfort zone they're willing to go to solve that problem
Software exists to create business value; it doesn't matter what it's written in or how it's hosted, so some systems will be around for a long time

Links:

Episode 39: Give 10 Bad Talks All in a Row and Then Get Fired

Corey Quinn — Wed, 05 Dec 2018 06:00:00 +0000

Do you like to hear yourself talk? Especially while on a stage and in front of a lot of people? How do you come up with ideas to talk about? What process do you use to build a conference talk or presentation?

Today, we’re talking to Matty Stratton of PagerDuty. His job involves building conference talks and finding ways to continuously improve them. Public speaking can be intimidating, so he shares some tips and tricks that have worked for him.

Some of the highlights of the show include:

Avoid creating something brand new for every event
Don’t tell flattering stories about things that happened to you; may be uplifting, but doesn't resemble reality
Failure stories are fantastic because people relate to making terrible decisions
Everyone who gives a talk panics, gets nervous, and thinks they’re about a sentence away from stammering and falling off the stage; almost never happens
Audience wants you to succeed because they're there to learn; no one is hoping a presenter messes up
Preparation is key; could build a talk at the last minute, but it would be much better, if you prepared for it
Don’t intentionally try to think of something; have conversations with people and listen to other talks to develop anecdotes, stories, and cold opens
Humor can be tricky; what you think is funny, other people might not
Make things memorable; show good ideas by showing bad ideas - it’s the ‘don't do this, do this instead’ model
Submit early and often, but submit appropriately; if you are always submitting stuff that’s inappropriate for an event, your stuff starts to be ignored
Sometimes, you may want to avoid slides that auto advance; if you trip over yourself: Stop, repeat, back up, take questions, etc.
Try not to read from notes or slides; takes the life and engagement out of the talk
People can only do one thing at a time - listen or read
Practice: Record yourself every time you practice and watch it; focus on blocking and tackling
You have about 45 seconds to grab people's interest before they look at their phone; get them engaged via a story, picture, or anecdote

Links:

Episode 38: Must be Willing to Defeat the JSON Heretics

Corey Quinn — Fri, 30 Nov 2018 17:45:00 +0000

Do you understand how tabs work? How spaces work? Are you willing to defeat the JSON heretics? Most people understand the power of the serverless paradigm, but need help to put it into a useful form. That’s where Stackery comes in to treat YAML as an assembly language. After all, no one programs processors like they did in the '80s with raw assembly routines and no one programs with C. Everyone is using a higher-level scripted or other programming language.

Today, we’re talking to Chase Douglas, co-founder and CTO of Stackery, which is serverless acceleration software where levels of abstraction empower you to move quickly. Stackery has an intricate binding model that gives you a visual representation - at a human logical level - of the infrastructure you defined in your application.

Some of the highlights of the show include:

Stackery builds infrastructures by using best practices with security applications
What's a VPC? Way to put resources into a Cloud account that aren’t accessible outside of that network; anything in that network can talk to each other
Lambda layers let developers create one Git layer that includes multiple functionality and put it in all functions for consistency and management
Git is an open-source amalgam of different programming languages that has grown and changed over time, but it has its own build system
Stackery created a PHP runtime functionality for Lambda; you don't want to run your own runtime - leave that up to a Cloud service provider for security reasons
Should you refactor existing Lambda functions to leverage layers? No, rebuild everything already built before re-architecting everything to use serverless
Many companies find serverless to be useful for their types of workloads; about 95% of workloads can effectively be engineered on a serverless foundation
Trough of Disillusionment or Gartner Hype Cycle: Stackery wants to re-engage and help people who have had challenges with serverless
Is DynamoDB considered serverless? Yes, because it’s got global replication
Puritanical (being able to scale down to zero) and practical approaches to the definition of serverless

Links:

Episode 37: Hiring in the Cloud “I assume CrowdStrike makes drones”

Corey Quinn — Wed, 21 Nov 2018 06:00:00 +0000

What’s hiring in the world of Cloud like? What are companies looking for in possible employees? What kind of career trajectory should applicants display?

Today, we’re talking to Don O’Neill, who has had an interesting career path and the archetype of who most companies want to hire. He’s been an independent contributor, platform leader, and Cloud consultant. Currently, Don is platform engineer manager at Articulate, an eLearning software solution for course authoring and eLearning development. He works with platform engineers to automate Blue Ocean pipelines with Docker, Terraform, and various Amazon Web Services (AWS) technologies, such as Elastic Beanstalk.

Some of the highlights of the show include:

Don reached out to his network to ask people that he had a professional relationship with about who was hiring and what challenges they faced
Don’s “Therapy”: Go to meet-ups to talk about DevOps topics; serves as a “I’ve-got-to-get-my-hiney-out-of-the-house-and-get-some-social-time”
Don’s journey from being a “wee lad in the industry” to a senior member/leader and giving back as a way to recognize those who helped him along the way
Hiring Horror Stories: People going through borderline ridiculous levels of hiring games and terrible interview paradigms
Companies sometimes look for something too specific - exact match instead of fuzzy match; they never have time to train, but time to look for a perfect unicorn
Articulate’s Hiring Process: Day 1 - Slack interview; Day 2 - Technical pieces; and Day 3 - Pairing with others
Articulate looks for people enthusiastic about technology, able to learn, and with emotional intelligence; company values independence, autonomy, and respect
Companies that spend several hours to make a hiring decision tend to have less success with those they hire
Cloud Certificates/Certifications: Can be valuable for applicants with no real-world experience; they don’t indicate how they’re going to work or learn
Applicants need to demonstrate a base level of knowledge; if they don’t have a skill set, they should start a project to learn about something - learning is fun
If you’re established in your career, reach out to someone just starting out to guide them
If you’re starting out in your career, reach out to people to talk about the next steps to take in your career (contact Corey or Don)

Links:

Episode 36: I'm Not Here to Correct Your English, Just Cloud Bills

Corey Quinn — Wed, 14 Nov 2018 12:14:00 +0000

Do you enjoy watching sports? Wear your favorite team or player’s jersey? Are you a fan who has shopped at Fanatics on the Cloud?

Today, we’re talking to Johnny Sheeley, director of Cloud engineering at Fanatics, which is a sports eCommerce business that manufactures and sells sports apparel. Fanatics runs Cloud engineering to provide a robust and reliable set of services by building and deploying applications on top of the Azure Data Lake Store (ADLS) platform.

Some of the highlights of the show include:

If you compete with Amazon, be ready for it to come after you; some companies avoid its Cloud perspective or go multi-Cloud (paranoia-based movement)
Focus on your ability to make your business function smoothly
Transition, migration, and abstraction may be painful, but should not stop work; paying for Cloud-agnostic technology may not be worth it
Challenges of governing use of Cloud resources to prevent mistakes/problems related to Fanatics’ security and budget
Data collected focuses on what’s trending up or down to select an instance type that calculates costs; remain flexible and be aware of what you pay
Natural instinct is to blame people; mistakes are made, especially when a human factor is introduced to an automated system
Creating a mindset that focuses on feature and detail-oriented is challenging
Cottage industry of code bases running in Big Data and other expensive realms
As a product continues to evolve and grow, governance comes along for the ride and AWS bills are streamlined
Will serverless, Lambda, and RDS change how Amazon charges in the future?
State of scale of AWS and developing a more palatable method for releases because people can’t keep up with them and stop paying attention
Two-Pizza Team: Amazon’s management philosophy that any team that works on a service should be able to be fed with two pizzas
Such small teams work quickly and have the freedom to fail, but Amazon has a reliability for the longevity of its different services

Links:

Episode 35: Metered Pricing: Everyone Hates That! Charge Based on Value

Corey Quinn — Wed, 07 Nov 2018 12:41:00 +0000

Did you know that you can now run Lambda functions for 15 minutes, instead of dealing with 5-minute timeouts? Although customers will probably never need that much time, it helps dispel the belief that serverless isn’t useful for some use cases because of such short time limits.

Today, we’re talking to Adam Johnson, co-founder and CEO of IOpipe. He understands that some people may misuse the increased timeframe to implement things terribly. But he believes the responsibility of a framework, platform, or technology should not be to hinder certain use cases to make sure developers are working within narrow constraints. Substantial guardrails can make developers shy away. With Lambda, they can do what they want, which is good and bad.

Some of the highlights of the show include:

Companies are using serverless as a foundation and for critical functions
Serverless can be painful in some areas, but gaps are going away
Investing in the Future: Companies doing lift-and-shift to AWS are looking at technology they should choose today that’s going to be prominent in 3 years
Serverless empowers new billing models and traces the flow of capital; companies can choose to make pricing more complicated or simplified
What value are you providing? Serverless can offer flexible pricing foundation
When something breaks, you need to be made aware of such problems; Amazon bill doesn’t change based on what IOpipe does, which is not true with others
Developers are the ones woken up and on call, so IOpipe focuses on providing them value and help; they are not left alone to figure out and fix problems
Serverless and event-driven applications offer a new type of instrumentation and observability to collect telemetry on every event
For serverless to go mainstream, AWS needs to up its observability level to gather data to answer questions
AWS, in the serverless space, needs to make significant progress on cold starts in other languages, and offer more visibility and easier deployment out of the box

Links:

Episode 34: Slack and the Safety Dance of Chaos Engineering

Corey Quinn — Wed, 31 Oct 2018 06:00:00 +0000

In the early days, angry nerd corners on the Internet viewed Slack and some of its predecessors as, “Oh, it’s just IRC. Now, you pay someone for it.” Many fell into that trap of wondering about what value such systems offered.The big differentiator? Slack is built as a collaborative business tool.

Today, we’re talking to Holly Allen, who helped make government software better while serving as the director of engineering at 18F. Now, she’s a senior engineering manager at Slack, a collaborative chat program where you can do most of your work through a rich platform of integrations. Holly enjoys taking a weird set of skills that make a computer do things and convincing people who know how to make computers do things do things.

Some of the highlights of the show include:

Safety engineering brings chaos and resilience engineering, incident management, and post-mortem processes together for resiliency and reliability
Slack strives to move really fast while being in complete control
Slack is primarily on AWS, but is working on a multi-Cloud strategy because if AWS is down, Slack still needs to work
Slack has a close relationship with AWS and is a collaborative company; it has immediate access to AWS staff anytime there’s a problem
Slack uses Terraform and Chef and working to determine if its production workflows in Kubernetes would be worthwhile
Disasterpiece Theater: Real scenario that might happen and surmise what will happen; don’t cause production issues, but teach Slack employees
Slack hires collaborative, empathetic people to create a collaborative environment where everyone works together toward a goal
Slack was firmly in a centralized operations model, but is transforming toward development teams to increase responsibility and service ownership
Slack doesn’t encourage remote work because it’s not in a position to put in that investment; day-to-day work happens in hallways and between desks
Slack sees itself as an enterprise software company; an enterprise software company must have enterprise software reliability, stability, and processes
Slack has thousands of servers, so events and disruptions happen more often; system needs to respond, react, and repair itself without human intervention

Links:

Episode 33: The Worst Manager I Ever Had Spoke Only In Metaphor

Corey Quinn — Wed, 24 Oct 2018 06:00:00 +0000

If you’ve been doing DevOps for the past 10-20 years, things have really changed in the industry. There’s no longer large pools of help desk support. People aren’t climbing around the data center and learning how to punch down cables and rack servers to gradually work their way up. Now, entry level DevOps jobs require about five years of experience. So, that’s where internships play a major role. But how can an internship program be set up for success? Where is the next generation of SREs or DevOps professionals coming from? Where do we find them?

Today, we’re talking to Fatema Boxwala, who has been an intern at Rackspace, Yelp, and Facebook. She’s a computer science student at the University of Waterloo in Canada, where she’s involved with the Women in Computer Science Committee and Computer Science Club. Occasionally, she teaches people about Python, Git, and systems administration.

Some of the highlights of the show include:

Mentors made Fatema’s intern experience positive for her; made site reliability and operations something she wanted to do
Academic paths don’t tend to focus on such fields as SRE, and interns tend to come exclusively from specific schools
Fatema’s school requires five internships to graduate and receive a degree; upper-year students are already very qualified professional software engineers
Companies don’t have time to train and want to find someone with an exact skill set; instead of hiring someone, they spend months with an unfilled position
Continuity Problem: You can’t train someone to be a systems administrator, if you aren’t willing to give them certain privileges due to inexperience
Use a low-stakes environment to train, where mistakes can be made; most systems aren’t on a critical path - don’t keep people away from contributing
If you have never broke production, that means either you’re lying or you’ve been in an environment that didn’t trust you to touch things that mattered
Internship should mimic the kind of work that everyone else is doing; give them responsibilities where their work has an impact
Bad mentors lead to bad internships; person in charge of your success doesn’t have the necessary skills; needs to be a good communicator, set expectations
As the intern, ask about possible outcomes of internship early on; mentors should be clear about expectations, feedback, and offers

Links:

Episode 32: Lambda School: A New Approach to “Hire Ed”

Corey Quinn — Wed, 17 Oct 2018 06:00:00 +0000

Are you interested in computer science? How would you like to go to school for free and learn what you need to in just a few months? Then, check out Lambda School!

Today, we’re talking to Ben Nelson, co-founder and CTO of Lambda School, which is a 30-week online immersive computer science academy. Lambda School has more than 500 students and takes a share of future earnings instead of traditional debt. So, it's free until students get a job.

Some of the highlights of the show include:

Bootcamps were created to address engineering shortages and quickly move people into technical careers
Lambda is not explicitly a bootcamp; its 30-week program gives students more instructions and more time spent on developing a portfolio
Lambda also makes time to cover computer science fundamentals; teaches C, Python, Django, and relational database - not just JavaScript
Employers appreciate the school’s in-depth and advanced approach, which results in repeat hires
Lambda avoids the typical reputation of traditional for-profit educational institutions by being mission-driven and knowing its investors want ROI
Lambda aligns its incentives with those of students; an income share agreement means the school doesn’t make money, unless students are successful
Lambda’s 7-month program is less of a risk for someone later in their career; some don't have capital to support their family while going to school for 4 years
Lambda incentivizes healthy financial habits; after two years of repayment, students can put that money into retirement, savings, and investments
5 Tracks Now Offered by Lambda: iOS development, UX, Full Stack Web development, data science, and Android development
Mastery Based Progression System: When you're learning something sequentially, where knowledge builds, you don't move on until you’ve mastered it
Lambda’s acceptance rate is around 5% and based on people who can keep up
Lambda works with different partner companies to help them find qualified graduates - people they want to hire

Links:

Episode 31: Hey Sam, wake up. It’s 3am, and time to solve a murder mystery!

Corey Quinn — Wed, 10 Oct 2018 06:28:00 +0000

Have you ever been on-call duty as an IT person or otherwise? Woken up at 3 a.m. to solve a problem? Did you have to go through log files or look at a dashboard to figure out what was going on? Did you think there has got to be a better way to troubleshoot and solve problems?

Today, we’re talking to Sam Bashton, who previously ran a premiere consulting partner with Amazon Web Services (AWS). Recently, he started runbook.cloud, which is a tool built on top of serverless technology that helps people find and troubleshoot problems within their AWS environment.

Some of the highlights of the show include:

Runbook.cloud looks at metrics to generate machine learning (ML) intelligence to pinpoint issues and present users with a pre-written set of solutions
Runbook.cloud looks at all potential problems that can be detected in context with how the infrastructure is being used without being annoying and useless
ML is used to do trend analysis and understand how a specific customer is using a service for a specific auto scaling group or Lambda functions
Runbook.cloud takes all aggregate data to influence alerts; if there’s a problem in a specific region with a specific service, the tool is careful to caveat it
Various monitoring solutions are on the market; runbook.cloud is designed for a mass market environment; it takes metrics that AWS provides for free and makes it so you don’t need to worry about them
Will runbook.cloud compete with or sell out to AWS? Amazon wants to build underlying infrastructure, other people to use its APIs to build interfaces for users
Runbook.cloud is sold through AWS Marketplace; it’s a subscription service where you pay by the hour and the charges are added to your AWS bill
Amazon vs. Other Cloud Providers: Work is involved to detect problems that address multiple Clouds; it doesn’t make sense to branch out to other Clouds
Runbook.cloud was built on top of serverless technology for business financial reasons; way to align outlay and costs because you pay for exactly what you use
Analysis paralysis is real; it comes down to getting the emotional toil of making decisions down to as few decision points as possible
Save money on Lambda; instead of using several Lambda functions concurrently, put everything into a single function using Go
AWS responds to customers to discover how they use its services; it comes down to what customers need

Links:

Episode 30: How to Compete with Amazon

Corey Quinn — Wed, 03 Oct 2018 06:00:00 +0000

Trying to figure out if Amazon Web Services (AWS) is right for you? Use the “quadrant of doom” to determine your answer. When designing a Cloud architecture, there are factors to consider. Any system you design exists for one reason - support a business. Think about services and their features to make sure they’re right for your implementation.

Today, we’re talking to Ernesto Marquez, owner and project director at Concurrency Labs. He helps startups launch and grow their applications on AWS. Ernesto especially enjoys building serverless architectures, automating everything, and helping customers cut their AWS costs.

Some of the highlights of the show include:

Amazon’s level of discipline, process, and willingness to recognize issues and fix them changed the way Ernesto sees how a system should be operated
Specialize on a specific service within AWS, such as S3 and EC2, because there are principles that need to be applied when designing an architecture
Sales and Delivery Cycle: Ernesto has a conversation with a client to discuss their different needs
Vendor Lock-in: Customers concerned about moving application to Cloud provider and how difficult it will be to move code and design variables elsewhere
For every service you include in your architecture, evaluate the service within the context of a particular business case
Identify failure scenarios, what can go wrong, and if something goes wrong, how it’s going to be remediated
CloudWatching detects events that are going to happen, and you can trigger responses for those events
Partnering with Amazon: Companies are pushing a multi-Cloud narrative; you gain visibility and credibility, but it’s not essential to be successful
Can you compete against Amazon? Depends on which area you choose
Expand product selection to grow, focus on user experience, and improve performance to compete against Amazon
MiserBot: Don’t freak out about your bill because Ernesto created a Slack chatbot to monitor your AWS costs

Links:

Episode 29: Future of Serverless: A Toy that will Evolve and Offer Flexibility

Corey Quinn — Wed, 26 Sep 2018 06:00:00 +0000

Are you a blogger? Engineer? Web guru? What do you do? If you ask Yan Cui that question, be prepared for several different answers.

Today, we’re talking to Yan, who is a principal engineer at DAZN. Also, he writes blog posts and is a course developer. His insightful, engaging, and understandable content resonates with various audiences. And, he’s an AWS serverless hero!

Some of the highlights of the show include:

Some people get tripped up because they don’t bring microservice practices they learned into the new world of serverless; face many challenges
Educate others and share your knowledge; Yan does, as an AWS hero
Chaos Engineering Meeting Serverless: Figuring out what types of failures to practice for depends on what services you are using
Environment predicated on specific behaviors may mean enumerating bad things that could happen, instead of building a resilient system that works as planned
API Gateway: Confusing for users because it can do so many different things; what is the right thing to do, given a particular context, is not always clear
Now, serverless feels like a toy, but good enough to run production workflow; future of serverless - will continue to evolve and offer more flexibility
Serverless is used to build applications; DevOps/IOT teams and enterprises are adopting serverless because it makes solutions more cost effective

Links:

Episode 28: Serverless as a Consulting Cash Register (now accepting Bitcoin!)

Corey Quinn — Wed, 19 Sep 2018 06:00:00 +0000

Is your company thinking about adopting serverless and running with it? Is there a profitable opportunity hidden in it? Ready to go on that journey?

Today, we’re talking to Rowan Udell, who works for Versent, an Amazon Web Services (AWS) consulting partner in Australia. Versent focuses on specific practices, including helping customers with rapid migrations to the Clouds and going serverless.

Some of the highlights of the show include:

Australia is experiencing an increase in developers using serverless tool services and serverless being used for operational purposes
Serverless seems to be either a brilliant fit or not quite ready for prime time
Misconceptions include keeping functions warm, setting up scheduled indications
Simon Wardley talked about how the flow of capital can be traced through an organization that has converted to serverless
Concept of paying thousands of dollars up front for a server is going away
Spend whatever you want, but be able to explain where the money is going (dev vs. prod); companies will re-evaluate how things get done
Serverless is either known as an evolution or revolution; transformative to a point
Winding up with a large number of shops where when something breaks, they don’t have the experience to fix it; gain practical experience through sharing
Seek developer feedback and perform testing, but know where and when to stop
With serverless, you have little control of the environment; focus on automated parts you do control
Serverless Movement: People have opinions and want you to know them
Understand continuum of options for running your application in the Cloud; learn pros and cons; and pick the right tool
Reconciliation between serverless and containers will need to play out; changes will come at some point
Blockchain + serverless + machine learning + Kubernetes + service mesh = raise entire seed round

Links:

Episode 27: What it Took for Google to Make Changes: Outages and Mean Tweets

Corey Quinn — Wed, 12 Sep 2018 06:00:00 +0000

Google Cloud Platform (GCP) turned off a customer that it thought was doing something out of bounds. This led to an Internet outrage, and GCP tried to explain itself and prevent the problem in the future.

Today, we’re talking to Daniel Compton, an independent software consultant who focuses on Clojure and large-scale systems. He’s currently building Deps, a private Maven repository service. As a third-party observer, we pick Daniel’s brain about the GCP issue, especially because he wrote a post called, Google Cloud Platform - The Good, Bad, and Ugly (It’s Mostly Good).

Some of the highlights of the show include:

Recommendations: Use enterprise billing - costs thousands of dollars; add phone number and extra credit card to Google account; get support contract
Google describing what happened and how it plans to prevent it in the future seemed reasonable; but why did it take this for Google to make changes?
GCP has inherited cultural issues that don’t work in the enterprise market; GCP is painfully learning that they need to change some things
Google tends to focus on writing services aimed purely at developers; it struggles to put itself in the shoes of corporate-enterprise IT shops
GCP has a few key design decisions that set it apart from AWS; focuses on global resources rather than regional resources
When picking a provider, is there a clear winner? AWS or GCP? Consider company’s values, internal capabilities, resources needed, and workload
GCP’s tendency to end service on something people are still using vs. AWS never ending a service tends to push people in one direction
GCP has built a smaller set of services that are easy to get started with, while AWS has an overwhelming number of services
Different Philosophies: Not every developer writes software as if they work at Google; AWS meets customers where they are, fixes issues, and drops prices
GCP understands where it needs to catch up and continues to iterate and release features

Links:

Episode 26: I’m not a data scientist, but I work for an AI/ML startup building on Serverless Containers

Corey Quinn — Wed, 05 Sep 2018 06:00:00 +0000

Do you deal with a lot of data? Do you need to analyze and interpret data? Veritone’s platform is designed to ingest audio, video, and other data through batch processes to process the media and attach output, such as transcripts or facial recognition data.

Today, we’re talking to Christopher Stobie, a DevOps professional with more than seven years of experience building and managing applications. Currently, he is the director of site reliability engineering at Veritone in Costa Mesa, Calif. Veritone positions itself as a provider of artificial intelligence (AI) tools designed to help other companies analyze and organize unstructured data. Previously, Christopher was a technical account manager (TAM) at Amazon Web Services (AWS); lead DevOps engineer at Clear Capital; lead DevOps engineer at ESI; Cloud consultant at Credera; and Patriot/THAAD Missile Fire Control in the U.S. Army. Besides staying busy with DevOps and missiles, he enjoys playing racquetball in short shorts and drinking good (not great) wine.

Some of the highlights of the show include:

Various problems can be solved with AI; companies are spending time and money on AI
Tasks can be automated that are too intelligent to write around simple software
Machine learning (ML) models are applicable for many purposes; real people with real problems and who are not academics can use ML
Fargate is instant-on Docker containers as a service; handles infrastructure scaling, but involves management expense
Instant-on works with numerous containers, but there will probably be a time when it no longer delivers reasonable fleet performance on demand
Decision to use Kafka was based on workload, stream-based ingestion
Veritone’s writes code that tries to avoid provider lock-in; wants to make an integration as decoupled as possible
People spend too much time and energy being agnostic to their technology and giving up benefits
If you dream about seeing your name up in lights, Christopher describes the process of writing a post for AWS
Pain Points: Newness of Fargate and unfamiliarity with it; limit issues; unable to handle large containers

Links:

Episode 25: Kubernetes is Named After the Greek God of Spending Money on Cloud Services

Corey Quinn — Wed, 29 Aug 2018 06:00:00 +0000

Google builds platforms for developers and strives to make them happy. There's a team at Google that wakes up every day to make sure developers have great outcomes with its services and products. The team listens to the developers and brings all feedback back into Google. It also spends a lot of time all over the world talking to and connecting with developer communities and showing stuff being worked on. It doesn't do the team any good to build developer products that developers don’t love.

Today, we’re talking to Adam Seligman, vice president of developer relations at Google, where he is responsible for the global developer community across product areas. He is the ears and voice for customers.

Some of the highlights of the show include:

Google tackles everything in an open source way: Shipping feedback, iteration, and building communities
Storytelling - the Tale of Kubernetes: in a short period of time, gone from being open source that Google spearheaded to something sweeping the industry
Rise of containerization inside Linux Kernel is an opportunity for Google to share container management technology and philosophy with the world
Google Next: Knative journey toward lighter-weight serverless-based applications; and GKE On-Prem, customers and teams working with Kubernetes running on premise
Innovation: When logging into GCP console, you can terminate all billable resources assigned to project and access tab for building by hand
GCP's console development strategy includes hard work on documentation, making things easy to use, and building thoughtfulness in grouping services
Google is about design goals, tradeoffs, and metrics; it’s about hyper scale and global footprint of requirements, as well as supporting every developer
Conception 1: Google builds HyperScale Reid-Centric user partitioned apps and don't build globally consistent data driven apps
Conception 2: Software engineers at the top Internet companies do the code and write amazing things instantly
12-Factor App: Opinions of how to architect apps; developers should have choices, but take away some cognitive and operating load complexity
Businesses are running core workloads on Google, which had to put atomic clocks in data centers and private fiber networking to make it all work
Perception that Google focuses on new things, rather than supporting what's been released; industry is on a treadmill chasing shiny things and creating noise
Industry needs to be welcoming and inclusive; a demand for software, apps, and innovation, but number of developers remains because everyone’s not included
Human vs. Technology: More investment and easier onboarding with technology and an obligation to build local communities
Goal: Take database complexity and start removing it for lots of use cases and simplify things for users to deal with replication, charting, and consistency issues
DevFest: Google has about 800 Google developer groups that do a lot of things to build local communities and write code together

Links:

Episode 24: Serverless Observability via the bill is terrible

Corey Quinn — Wed, 22 Aug 2018 06:00:00 +0000

What is serverless? What do people want it to be? Serverless is when you write your software, deploy it to a Cloud vendor that will scale and run it, and you receive a pay-for-use bill. It’s not necessarily a function of a service, but a concept.

Today, we’re talking to Nitzan Shapira, co-founder and CEO of Epsagon, which brings observability to serverless Cloud applications by using distributed tracing and artificial intelligence (AI) technologies. He is a software engineer with experience in software development, cyber security, reverse engineering, and machine learning.

Some of the highlights of the show include:

Modern renaissance of “functions as a service” compared to past history; is as abstracted as it can be, which means almost no constraints
If you write your own software, ship it, and deploy it - it counts as serverless
Some treat serverless as event-driven architecture where code swings into action
When being strategic to make it more efficient, plan and develop an application with specific and complicated functioning
Epsagon is a global observer for what the industry is doing and how it is implementing serverless as it evolves
Trends and use cases include focusing on serverless first instead of the Cloud
Economic Argument: Less expensive than running things all the time and offers ability to trace capital flow; but be cautious about unpredictable cost
Use bill to determine how much performance and flow time has been spent
Companies seem to be trying to support every vendor’s serverless offering; when it comes to serverless, AWS Lambda appears to be used most often
Not easy to move from one provider to another; on-premise misses the point
People starting with AWS Lambda need familiarity with other services, which can be a reasonable but difficult barrier that’s worth the effort
Managing serverless applications may have to be done through a third party
Systemic view of how applications work focuses on overall health of a system, not individual function
Epsagon is headquartered in Israel, along with other emerging serverless startups; Israeli culture fuels innovation

Links:

Episode 23: Most Likely to be Misunderstood: The Myth of Cloud Agnosticism

Corey Quinn — Fri, 10 Aug 2018 06:00:00 +0000

It is easy to pick apart the general premise of Cloud agnosticism being a myth. What about reasonable use cases? Well, generally, when you have a workload that you want to put on multiple Cloud providers, it is a bad idea. It’s difficult to build and maintain. Providers change, some more than others. The ability to work with them becomes more complex. Yet, Cloud providers rarely disappoint you enough to make you hurry and go to another provider.

Today, we’re talking to Jay Gordon, Cloud developer advocate for MongoDB, about databases, distribution of databases, and multi-Cloud strategies. MongoDB is a good option for people who want to build applications quicker and faster but not do a lot of infrastructural work.

Some of the highlights of the show include:

Easier to consider distributed data to be something reliable and available, than not being reliable and available
People spend time buying an option that doesn’t work, at the cost of feature velocity
If Cloud provider goes down, is it the end of the world?
Cloud offers greater flexibility; but no matter what, there should be a secondary option when a critical path comes to a breaking point
Hand-off from one provider to another is more likely to cause an outage than a multi-region single provider failure
Exclusion of Cloud Agnostic Tooling: The more we create tools that do the same thing regardless of provider, there will be more agnosticism from implementers
Workload-dependent where data gravity dictates choices; bandwidth isn’t free
Certain services are only available on one Cloud due to licensing; but tools can help with migration
Major service providers handle persistent parts of architecture, and other companies offer database services and tools for those providers
Cost may/may not be a factor why businesses stay with 1 instead of multi-Cloud
How much RPO and RTO play into a multi-Cloud decision
Selecting a database/data store when building; consider security encryption

Links:

Episode 22: The Chaos Engineering experiment that is us-east-1

Corey Quinn — Wed, 08 Aug 2018 06:00:00 +0000

Trying to convince a company to embrace the theory and idea of Chaos Engineering is an uphill battle. When a site keeps breaking, Gremlin’s plan involves breaking things intentionally. How do you introduce chaos as a step toward making things better?

Today, we’re talking to Ho Ming Li, lead solutions architect at Gremlin. He takes a strategic approach to deliver holistic solutions, often diving into the intersection of people, process, business, and technology. His goal is to enable everyone to build more resilient software by means of Chaos Engineering practices.

Some of the highlights of the show include:

Ho Ming Li previously worked as a technical account manager (TAM) at Amazon Web Services (AWS) to offer guidance on architectural/operational best practices
Difference between and transition to solutions architect and TAM at AWS
Role of TAM as the voice and face of AWS for customers
Ultimate goal is to bring services back up and make sure customers are happy
Amazon Leadership Principles: Mutually beneficial to have the customer get what they want, be happy with the service, and achieve success with the customer
Chaos Engineering isn’t about breaking things to prove a point
Chaos Engineering takes a scientific approach
Other than during carefully staged DR exercises, DR plans usually don’t work
Availability Theater: A passive data center is not enough; exercise DR plan
Chaos Engineering is bringing it down to a level where you exercise it regularly to build resiliency
Start small when dealing with availability
Chaos Engineering is a journey of verifying, validating, and catching surprises in a safe environment
Get started with Chaos Engineering by asking: What could go wrong?
Embrace failure and prepare for it; business process resilience
Gremlin’s GameDay and Chaos Conf allows people to share experiences

Links:

Episode 21: Remember when RealNetworks used to-- BUFFERING

Corey Quinn — Wed, 01 Aug 2018 06:00:00 +0000

Are you about to head off to college? Interested in DevOps and the Cloud? Is there a good way for someone like you who is starting out in the world of technology to absorb the necessary skills? The Open Source Lab (OSL) at Oregon State University (OSU) is one program that helps students and serves as a career accelerator. OSL is a unicorn because OSU is willing to invest in open source.

Today, we’re talking to Lance Albertson, director of OSL at OSU. OSL does a variety of projects to provide private Clouds that are neutrally hosted on its premises. The lab also gives undergraduate students hands-on experience with DevOps skills, including dealing with configuration management, deploying applications, learning how applications deploy, working with projects, and troubleshooting issues. OSL is for any student who has a general interest or passion for it, and a willingness to learn.

Some of the highlights of the show include:

Workflow focuses on what students need to learn about Linux and giving access to various repos; then they experience the lab’s configuration management suite
Interview Process: Put out a posting, student submits an application online, each candidate is reviewed, student is given a screening quiz,
If a student passes the screening process, they are brought in for an in-person interview for personality and technical questions
Students tend to initially have the least amount of experience and most difficulty with a repository that has multiple people committing to it and dealing with PRs
Spinning up VMs and understanding how configuration management is connected, how services communicate, and how to set up an application
Round-Robins and System Sprint Meetings: Focus on discussing and documenting processes, issues, suggestions, comments, and other information
Younger students are mentored by Lance and the older students; every generation has to evolve because the environment and industry evolve
OSL made OpenStack work on POWER8, PowerPC, and PowerPC little-endian; gateway into Cloud - having OpenStack instance to offer services
Vast majority of OSL’s revenue comes from donations; no direct support from the university; finding companies to serve as sponsors is beneficial to all
Future of OSL: Providing more Cloud-like services; creating a more internal, private Cloud’ and containerized ways of running or deploying applications

Links:

Episode 20: The Wizard of AWS

Corey Quinn — Wed, 25 Jul 2018 06:58:00 +0000

Today, we’re talking to Jeff Barr, vice president and chief evangelist at Amazon Web Services (AWS). He founded the AWS Blog in 2004 and has written more than 2,900 posts for it and another 1,100 for his personal blog. As chief evangelist, Jeff strives to explain the benefits of Cloud computing and Web services to anyone who will listen.

Jeff is the voice of AWS. He does what he does best - exploits his superpower of explaining technology in ways that people can understand it. Jeff tries to be the same person all the time. He loves to meet people and go out of his way to say “Hello.” So, if you see him at re:Invent, say “Cheese” and take a selfie with him!

Some of the highlights of the show include:

Jeff uses AWS Workspaces for his blog; one of Jeff’s blogging principles is to not take anybody else's word for anything to the absolute best of his technical ability
Zero Client: Jeff has no rotating hardware, disk drives, just a zero client; wherever he is, it's the same workspace
AWS has something for everyone; it build things in response to customers’ questions, requests, and feedback
Naming Services and Products: Is it helpful? Is it descriptive? Does it have any hidden meanings?
Amazonian DNA and Dog Friendly Workspace: Jeff went from super fearful to accepting, to now thinking of dogs as incredible creations because they add fun and excitement to the office
As part of hiring, each interviewer is assigned Amazon leadership principles (LPs) to ask questions that measure a candidate against those LPs
What is the secret to getting hired at Amazon? Study the LPs to understand what they're about and be able to express your philosophies and history with LPs
re:Invent makes sure customers understand services - What is it? What does it do? How do they put it to work? What are the best use cases for it?
Things can never be too simple; you start from zero, put a lot of different things in there, and then you need the feedback to build in simplicity
AWS is following a more on-demand approach than traditional reserve instances; it opens the door to being used in a lot of ways
AWS does a lot of work before a launch to make sure it’s got infrastructure, scaling, monitoring, and capacity in place
If you are a customer, talk to AWS and let them know what they're doing right or wrong; write a blog post, tweet about it, share it with them in some way
Is the breadth of product offerings from AWS too vast? Is it offering too many things?
AWS was not explicit about where it was going with Cloud computing or do analyses or projections about it; it simply launched SQS and let it speak for itself
Customer feedback shapes what Amazon works on; customers share and then AWS re-prioritizes to make sure it’s delivering the right thing at the right time
Remember: It's not just bits and bytes, it's about the organic life form

Links:

Jeff Barr on Twitter

Jeff Barr on LinkedIn

AWS

AWS Blog

Jeff Barr’s Blog

Amazon Machine Images

Zero Client

AWS Workspaces

AWS Lambda

Amazon Leadership principles

re:Invent

The Robot Uprising Will Have Very Clean Floors

Serverlessly Storing My Dad Jokes in a Dadabase

Days Until re:Invent

Episode 19: I want to build a world spanning search engine on top of GCP

Corey Quinn — Thu, 19 Jul 2018 00:12:00 +0000

Some companies that offer services expect you to do things their way or take the highway. However, Google expects people to simply adapt the tech company’s suggestions and best practices for their specific context. This is how things are done at Google, but this may not work in your environment.

Today, we’re talking to Liz Fong-Jones, a Senior Staff Site Reliability Engineer (SRE) at Google. Liz works on the Google Cloud Customer Reliability Engineering (CRE) team and enjoys helping people adapt reliability practices in a way that makes sense for their companies.

Some of the highlights of the show include:

Liz figures out an appropriate level of reliability for a service and how a service is engineered to meet that target
Staff SRE involves implementation, and then identifying and solving problems
Google’s CRE team makes sure Google Cloud customers can build seamless services on the Google Cloud Platform (GCP)
Service Level Objectives (SLOs) include error budgets, service level indicators, and key metrics to resolve issues when technology fails
Learn from failures through instant reports and shared post-mortems; be transparent with customers and yourself
GCP: Is it part of Google or not? It’s not a division between old and new.
Perceptions and misunderstandings of how Google does things and how it’s a different environment
Google’s efforts toward customer service and responsiveness to needs
Migrating between different Cloud providers vs. higher level services
How to use Cloud machine learning-based products
GCP needs to focus on usability to maintain a phase of growth
Offer sensible APIs; tear up, turn down, and update in a programmatic fashion
Promotion vs. Different Job: When you’ve learned as much as you can, look for another team to teach something new
What is Cloud and what isn’t? Cloud deployments require SRE to be successful but SREs can work on systems that do not necessarily run in the Cloud.

Links:

Episode 18: Sitting on the curb clapping as serverless superheroes go by

Corey Quinn — Wed, 11 Jul 2018 06:00:00 +0000

What’s serverless? Are you serverless now? Is going from enterprise to serverless a natural evolution? Or, is it a “that was fun, now let’s go ride our bikes” moment? Is serverless “just a toy?” Is it a wide and varied ecosystem, or is it Lambda plus some other randos? What's up with serverless vs. containers?

Today, Forrest Brazeal is here to answer those questions and discuss pros and cons of serverless. He was a senior Cloud architect prior to joining Trek10. Forrest spent several years leading AWS and serverless engineering projects at Infor. He understands the challenges faced by enterprises moving to the Cloud and enjoys building solutions that provide maximum business value at a minimal cost.

Some of the highlights of the show include:

Bimodality: Backend development going away and being replaced by managed services; undifferentiated items are being moved to the Cloud
Serverless is application designs with “Backend as a Service” (BaaS) and/or “Functions as a Service” (FaaS) platforms; everything is managed for you
AWS Lambda: Is it today’s trend or a bias that everyone is using it; Lambda makes up 80% of current FaaS adoption
Serverless Ecosystem: You can build it however you want, and you’re doing it right; but don’t take that at face-value; no two Lambda environments are alike
Cloud services at this scale have not been knitted together to form applications that are serving major workloads; best practices need to be established
Native Cloud providers will consolidate, and individual frameworks will be created with components of application stacks tied together to build systems
Serverless vs. Containers: No need for disparity - we can learn to get along; people use containers because it is easier than going serverless
Serverless Heroes series features people thinking out-of-the-box and helps identify emerging trends; serverless is growing, and it’s not just about startups
Went from working with a Sharpie to Procreate for the FaaS and Furious cartoon series; serverless component of process is for invoicing
Changes? Packaging to handle sharing; more knobs on console; unified process needed because too many building own workflow and tooling
Certification: Proof-positive that you know what you’re talking about or is it questionable value if not backing up expertise in the real world?

Links:

Episode 17: Pouring Kubernetes on things with reckless abandon

Corey Quinn — Wed, 04 Jul 2018 06:00:00 +0000

DevOps as a service describes what Reactive Ops is trying to do, who it’s trying to help, and what problems it’s trying to solve. It’s passion to deliver service where human beings help other human beings is done through a group of engineers who are extremely good at solving problems.

Sarah Zelechoski is the vice president of engineering at Reactive Ops, which defines the world’s problems and solves them by pouring Kubernetes on top of them. The team focuses on providing expert-level guidance and a curated framework using Kubernetes and other open source tools. Sarah's greatest passion is helping others, which encompasses advocating for engineers and rekindling interest in the lost art of service in the tech space.

Some of the highlights of the show include:

Kubernetes is changing the way people work; it offers a way to release a product, provide access to it, and behaviors when you deploy it
Any person/business can use Kubernetes to mold their workflow
Kubernetes is complex and has sharp edges; it has only recently become productive because of its community finding and reporting issues
Business value of deploying Kubernetes to a new environment: Flexibility and uniform system of management; and it can provide a context shift
Implementation Challenges with Workshops/Tutorials: Valuable entry level strategy for people learning Kubernetes; but the translation is not easy
About 85% of the work Reactive Ops does is helping its customers get on to Kubernetes is spent on application architecture
If thinking about moving to Kubernetes, how well will your current applications translate? Do you want to start over from scratch?
Value in paying someone to do something for you
Using Defaults: Try initially until you realize what you need; Kubernetes gives you options, but it’s a challenging path to go from defaults to advanced
Deploying a workload between all major Cloud providers is possible, but there are challenges in managing multiple regions or locations
Cluster Ops: Managed Kubernetes clusters where Reactive Ops stays on the map, watches them, and puts them on pager, so you can continue your work without having to worry

Links:

Episode 16: There are Still Servers, but We Don't Care About Them

Corey Quinn — Wed, 27 Jun 2018 06:00:00 +0000

Are you interested in going beyond basic monitoring and visibility? Need tools to build and operate serverless applications and extract business intelligence? IOpipe provides extended visibility and metrics around AWS Lambda, including profiling, core dumps, and incoming input events.

Today, we’re talking to Erica Windisch, who is the founder and CTO of IOpipe. She brings her experience in building developer and operational tooling to serverless applications. Erica also has more than 17 years of experience designing and building Cloud infrastructure management solutions. She was an early and longtime contributor to OpenStack and maintainer of the Docker project.

Some of the highlights of the show include:

Nomenclature Battle: Serverless vs. stateless
Building a window of visibility into Lambda: Talking to users and assessing needs/pain points
Observability of the infrastructure: Necessary evil to get to automated healing
Using Lambda at significant levels of scale; some companies grow usage, others go all in right away
Current state of Lambda ecosystem
Is Lambda stable? Indications and no formal SLA
How issues manifest and are exposed
Trends include cold starts, hours-long failures, and multiple function evokes
Infrastructure powering IOpipe: Lambda issues may impact performance of monitoring system, but IOpipe is not necessarily dependent on Lambda
Future of Lambda: Builds applications a specific way, but there are limitations
What would Erica change about Lambda? Run function and define handlers
Lambda functions can be difficult to understand; some developers do not have familiarity and create bottlenecks
Capacity limits around Lambda can be difficult to establish

Links:

Episode 15: Nagios was the Original Call of Duty

Corey Quinn — Wed, 20 Jun 2018 06:00:00 +0000

Let’s chat about the Cloud and everything in between. The people in this world are pretty comfortable with not running physical servers on their own, but trusting someone else to run them. Yet, people suffer from the psychological barrier of thinking they need to build, design, and run their own monitoring system. Fortunately, more companies are turning to Datadog.

Today, we’re talking to Ilan Rabinovitch, Datadog’s vice president of product and community. He spends his days diving into container monitoring metrics, collaborating with Datadog’s open source community, and evangelizing observability best practices. Previously, Ilan led infrastructure and reliability engineering teams at various organizations, including Ooyala and Edmunds.com. He’s active in the open source and DevOps communities, where he is a co-organizer of events, such as SCALE and Texas Linux Fest.

Some of the highlights of the show include:

Datadog is well-known, especially because it is a frequent sponsor
More organizations know their core competency is not monitoring or managing servers
Monitoring/metrics is a big data problem; Datadog takes monitoring off your plate
Alternate ways, other than using Nagios, to monitor instances and regenerate configurations
Datadog is first to identify patterns when there is a widespread underlying infrastructure issue
Trends of moving from on-premise to Cloud; serverless is on the horizon
How trends affect evolution of Datadog; adjusting tools to monitor customers’ environments
Datadog’s scope is enormous; the company tries to present relevant information as the scale of what it’s watching continues to grow
Datadog’s pricing is straightforward and simple to understand; how much Cloud providers charge to use Datadog is less clear
Single Pane of Glass: Too much data to gather in small areas (dashboards)
Why didn’t monitoring catch this? Alerts need to be actionable and relevant
How to use Datadog’s workflow for setting alerts and work metrics
Datadog’s first Dash user conference will be held in July in New York; addresses how to solve real business problems, how to scale/speed up your organization

Links:

Episode 14: Cheslocked and loaded

Corey Quinn — Wed, 13 Jun 2018 06:00:00 +0000

Do you need data captured that let you know when things don’t look quite right? Need to identify issues before they become major problems for your organization? Turn to Threat Stack, which has Cloud issues of its own, and helps its customers with their Cloud issues.

Today, I’m talking to Pete Cheslock, who runs technical operations at Threat Stack, which handles security monitoring, alerting, and remediation. The company uses Amazon Web Services (AWS), but its customer base can run anywhere.

Some of the highlights of the show include:

Challenges Threat Stack experienced with AWS and how it dealt with them
Threat Stack helps companies improve their security posture in AWS
Security shouldn’t be an issue, if providers do their job; shared responsibility
Education is needed about what matters regarding security, avoiding mistakes
Cloud is still so new; not many people have abroad experience managing it
Scanning customer accounts against best practices to identify risks
Threat Stack’s scanning tool is worthwhile, but most tools lack judgement and perspective
Threat Stack offers context between host- and Cloud-based events; tying data together is the secret sauce
You shouldn’t have to pay a bunch of money to have a robust security system
Good operations is good security; update, patch, track, and perform other tasks
Lack of validation about what services are going to be a successful or not
Vendor Lock-in: Understand your choices when building your system
Pervasiveness and challenge of containerization and Kubernetes
Cloud reduces cycle time and effort to bring a product to market
Amazon is a game changer with what it allows you to do and solve problems

Links:

Episode 13: Serverlessly Storing my Dad Jokes in a Dadabase

Corey Quinn — Wed, 06 Jun 2018 00:00:00 +0000

Aurora, from Amazon Web Services (AWS), is a MySQL-compatible service for complex database structures. It offers capabilities and opportunities. But with Aurora, you’re putting a lot of trust in AWS to “just work” in ways not traditional to relational database services (RDS).

David Torgerson, Principal DevOps Engineer at Lucidchart, is a mystery wrapped in an enigma and virtually impossible to Google. He shares Lucidchart’s experience with migrating away from a traditional RDS to Aurora to free up developer time.

Some of the highlights of the show include:

Trade off of making someone else partially responsible for keeping your site up
Lucidchart’s overall database costs decreased 25% after switching to Aurora
Aurora unknowns: What is an I/Op in Aurora? When you write one piece of data, does it count as six I/Ops?
Multi-master Aurora is coming for failover time and disaster recovery purposes
Aurora drawbacks: No dedicated DevOps, increased failover time, and misleading performance speed
Providers offer ways to simplify your business processes, but not ways to get out of using their products due to vendor and platform lock-in
Lucidchart is skeptical about Aurora Serverless; will use or not depending on performance

Links:

Episode 12: Like Normal Cloud Services, but More Depressing

Corey Quinn — Wed, 30 May 2018 06:00:00 +0000

Does your job challenge and motivate you? Does it utilize your skills? Or, are you ready to go job hunting? Do you want an awesome job that is a resume booster? Companies should be supportive of their employees finding a job that matches their skills and interests. Also, when hiring, companies should offer thoughtful processes for interviews.

Today, I’m talking to Sarah Withee, a polyglot software engineer, mentor, teacher, and robot tinkerer. Sarah went job hunting, and after several job interviews, she finally found a job that made her super happy at Arcadia Healthcare Solutions. Sarah compares the interview processes she experienced at big name tech companies that offer Cloud services.

Some of the highlights of the show include:

Companies sometimes lose sight that even interview interactions need to be a two-way sale
Interviews often involve talking to many people; and if several are bad, that forms a negative impression of the company
Companies need to provide interview training and follow the same standards
Don’t farm out challenging or unfamiliar issues when interviewing candidates
Sarah is very competent, but she is new to Cloud platforms; she is like a sponge, who enjoys learning and having a bare knowledge of new technology
How HIPAA regulations impact Sarah’s learning and software engineering work; she has to be more aware of security and safety of healthcare data
Being a teacher and mentor affects how Sarah learns new things; everybody learns slightly differently
In the Cloud space, know which direction you want to go and start with simpler things to learn the basics; focus on what is relevant to what you are working on

Links:

Episode 11: Hickory Dickory Docker

Corey Quinn — Wed, 23 May 2018 00:00:00 +0000

Docker went from being a small startup to an enterprise company that changed the way people think about their infrastructure to now, where its relevance is somewhat minimal. The conversation is no longer around the container level. Docker has become commonplace.

Today, we’re talking to Jérôme Petazzoni, formerly of Docker. While he was with the company for about 8 years, Docker definitely experienced a roller coaster ride.

Some of the highlights of the show include:

Amount of work conducted on the enterprise vs. community editions
Docker was so widely adopted because its core technology was open source
Challenge is to build a viable business and revenue model for the long run
Similarities between Docker and Red Hat open source platforms
Docker went from six people working in a garage to having a few hundred employees and $1.3 billion valuation
Changes happened, but they were gradual; the changes were necessary to be a profitable and sustainable company
Contingent of internal and external people believed that Docker was the answer for whatever problem surfaced; Docker would save you, but not always
Balancing Act: Pushing forward with a correct message and regulating enthusiasm
Networking and Docker for dummies; confusion and problems of things not working as expected have been resolved
Things will continue to shift; Kubernetes and the orchestration battle
What was unthinkable, could happen by companies pushing the envelope and making progress
Will who you have as your Cloud provider stop mattering? It depends.
All major Cloud providers plan to offer managed Kubernetes services and what Jérôme thinks of them
Jérôme’s opinion on whether Kubernetes will follow this same path as Docker
What does the road ahead look like for infrastructure automation? There is potential and lots of best practices in Cloud environments.

Links:

Episode 10: Education is Not Ready for Teacherless

Corey Quinn — Wed, 16 May 2018 06:00:00 +0000

Like migrating caribou, you tend to follow the trends of what clients are doing, which dictates what you work on as a consultant.

Today, we’re talking to Lynn Langit, an independent Cloud architect. She is an AWS Community Hero, Google Cloud developer expert, and former Microsoft MVP. Lynn is a lifelong learner, and she has worked broad and deep across all three large providers. These days, she works mostly with Google Cloud and AWS, rather than Azure, because that’s what her clients are using.

Some of the highlights of the show include:

Differences between the West Coast and global use of Cloud
Education is key; Lynn is th co-founder of Teachingkidsprogramming.org
Lynn helped create curriculum and resources for school-age children; even her young daughter taught classes on how to code
Training for teachers was also needed, so TKP Labs was formed to offer fee-based teacher and developer training
Lynn started with classroom training, but has transitioned to online learning
Lynn is focusing on Big Data projects and using tools to solve real-world problems
Pre-processing and batching data, but not streaming it
AWS, Azure, and Google Cloud are all coming out with Big Data-oriented tools
Companies need to understand when the market is ready to accept a new paradigm; in the data world, change is more slow than in the programming world
If you touch a database and get burned, you are not willing to use it again; or you may have never tried to archive your data; hire a consultant to help you
Machine learning APIs give customers value quickly; review them before building custom models
Migrating data can be a costly project and restricts where the data lives
As Cloud proliferates, how will that impact technical education? Lynn’s Cloud for College Students to the rescue!
Shift from interactive to unidirectional, one-to-many learning styles; the Cloud is ready for serverless, but education is not ready for teacherless
Road that many of us walked to get to technical skills no longer exists; how to become a modern technologist
Ageism: By age 40, you are considered a manager or useless; don’t be afraid to learn something new

Links:

Episode 9: Cloud Coreyography

Corey Quinn — Wed, 09 May 2018 06:00:00 +0000

Microsoft has experienced a renaissance. By everything that we've seen coming out of Microsoft over the past few years, it feels like the company is really walking the walk. Instead of just talking about how it’s innovative, it’s demonstrating that. Microsoft has been on an amazing journey, making the progression from telling customers what they need to listening to them and responding by building what they ask for.

Today, we’re talking to Corey Sanders, Corporate Vice President of Azure Compute at Microsoft.

Some of the highlights of the show include:

Customers are asking for Microsoft to help them through support and enabling platforms
Storytelling efforts through advocates, who play a double role – engaging and defending Microsoft
Customers moving to the Cloud are focused on a continuum and progression; they have stuff to move from one location to another and want all the benefits–better agility, faster startup time, etc.
Virtual serial console into existing VMs; this is how people are using this and Microsoft is going to, if not encourage this behavior, at least support it
Microsoft is the only Cloud with a single-instance SLA
Serial consoles: Windows' has seen less usage, partly due to operational aspects of Windows vs. Linux. It's not a GUI; it's scripting.
Does the operating system matter? From a Cloud perspective, it shouldn't have to matter; you should be able to deploy it the way you want
Edge enables much more complex and segregated scenarios; that combination with cognitive searches running locally will make it accessible anywhere
Branding challenge as customers start to notice that devices are smarter and more complex; will they lose awareness that Microsoft Azure is powering most of these things - they shouldn’t care
An awareness of not just what's possible, but what's coming; the democratization of AI
Education and fear gap of trying something new and taking that first step; make products and services stupid and simple to use
Customers return to add cognitive services and AI capabilities to existing, running deployments, environments, and applications
Multi-Cloud solutions can be successful, but there's a caveat; they’re actually built on a service-by-service perspective
Azure Stack, offers consistency, but some people may place blame on it for poor data center management practices; some expectations and regulations may be frustrating to some customers, but lets Microsoft offer a consistent experience
Freedom and flexibility have been challenges for Microsoft and other products for private Clouds
What people need to understand about Azure, including from a durability and reliability experience
To some extent, scale becomes a necessary prerequisite for some applications
Microsoft has taken many steps and is the leader in various areas

Links:

ReactiveOps
Microsoft Azure
Corey Sanders on Twitter
The Robot Uprising Will Have Very Clean Floors
Kubernetes
Cassandra
Azure Stack

Episode 8: A Corporate Prisoner's Dilemma

Corey Quinn — Wed, 02 May 2018 06:00:00 +0000

Have you dabbled with IT infrastructure in AWS? Have you been through the process of AWS partnership? Does being an AWS partner add value? Amazon seeks partners that helps drive its business, goals, and value.

Today, we’re talking to Justin Brodley, the vice president of Cloud engineering at Ellie Mae. He has been through the AWS partnership process and shares his thoughts about it. He encourages you to find the right partner for your business!

Some of the highlights of the show include:

Different levels and types of AWS partnerships
Shakedown vs. opportunity method for new leads; lead generation expectations
Amazon’s improvements eroding business models
Partners trying to pivot, but not exclusive to AWS
Whether to invest in multi-Cloud
Amazon can’t scale its sales team to handle everybody; views partner program as an extension of its salesforce
Your company is important and you’re spending a lot of money, but Amazon may not care about you; partner market fills that gap and makes you feel important
Corporate prisoner’s dilemma: Your tech company offers something that Amazon doesn’t; but what about when Amazon does offer it?
Competitors’ horizontal move to become more diversified
Amazon expects partners to offer products and services that it cannot offer yet
If partners fail, Amazon decides to do it and do it better
Is Amazon’s best interest geared toward its partners or you and your customers?
Amazon needs to give incentives and support partners

Links:

Episode 7: The Exact Opposite of a Job Creator

Corey Quinn — Wed, 25 Apr 2018 00:00:00 +0000

Monitoring in the entire technical world is terrible and continues to be a giant, confusing mess. How do you monitor? Are you monitoring things the wrong way? Why not hire a monitoring consultant!

Today, we’re talking to monitoring consultant Mike Julian, who is the editor of the Monitoring Weekly newsletter and author of O’Reilly’s Practical Monitoring. He is the voice of monitoring.

Some of the highlights of the show include:

Observability comes from control theory and monitoring is for what we can anticipate
Industry’s lack of interest and focus on monitoring
When there’s an outage, why doesn’t monitoring catch it?” Unforeseen things.
Cost and failure of running tools and systems that are obtuse to monitor
Outsource monitoring instead of devoting time, energy, and personnel to it
Outsourcing infrastructure means you give up some control; how you monitor and manage systems changes when on the Cloud
CloudWatch: Where metrics go to die
Distributed and Implemented Tracing: Tracing calls as they move through a system
Serverless Functions: Difficulties experienced and techniques to use
Warm vs. Cold Start: If a container isn't up and running, it has to set up database connections
Monitoring can't fix a bad architecture; it can't fix anything; improve the application architecture
Visibility of outages and pain perceived; different services have different availability levels

Links:

Episode 6: The Robot Uprising Will Have Very Clean Floors

Corey Quinn — Wed, 18 Apr 2018 06:00:00 +0000

How many of you are considered heroes? Specifically, in the serverless Cloud, Twitter, and Amazon Web Services (AWS) communities? Well, Ben Kehoe is a hero.

Ben is a Cloud robotics research scientist who makes serverless Roombas at iRobot. He was named an AWS Community Hero for his contributions that help expand the understanding, expertise, and engagement of people using AWS.

Some of the highlights of the show include:

Ben’s path to becoming a vacuum salesman
History of Roomba and how AWS helps deliver current features
Roombas use AWS Internet of Things (IoT) for communication between the Cloud and robot
Boston is shaping up to be the birthplace of the robot overlords of the future
AWS IoT is serverless and features a number of pieces in one service
Robot rising of clean floors
AWS Greengrass, which deploys runtimes and manages connections for communication, should not be ignored
Creating robots that will make money and work well
Roomba’s autonomy to serve the customer and meet expectations
Robots with Cloud and network connections
Competitive Cloud providers were available, but AWS was the clear winner
Serverless approach and advantages for the intelligent vacuum cleaner
Future use of higher-level machine learning tools
Common concern of lock-in with AWS
Changing landscape of data governance and multi-Cloud
Preparing for migrations that don’t happen or change the world
Data gravity and saving vs. spending money

Links:

Episode 5: The Last Mainframe with a Kickstart and a Double Clutch

Corey Quinn — Wed, 11 Apr 2018 06:00:00 +0000

How are companies evolving in a world where Cloud is on the rise? Where Cloud providers are bought out and absorbed into other companies?

Today, we’re talking to Nell Shamrell-Harrington about Cloud infrastructure. She is a senior software engineer at Chef, CTO at Operation Code, and core maintainer of the the Habitat open source product. Nell has traveled the world to talk about Chef, Ruby, Rails, Rust, DevOps, and Regular Expressions.

Some of the highlights of the show include:

Chef is a configuration management tool that handles instance, files, virtual machine container, and other items.
Immutable infrastructure has emerged as the best of practice approach.
Chef is moving into next gen through various projects, including one called, Compliance - a scanning tool.
Some people don’t trust virtualization.
Habitat is an open source project featuring software that allows you to use a universal packaging format.
Habitat is a run-time, so when you run a package on multiple virtual machines, they form a supervisor ring to communicate via leader/follower roles.
Deploying an application depends on several factors, including application and infrastructure needs.
It is possible to convert old systems with old deployment models to Habitat.
Habitat allows you to lift a legacy application and put it into that modern infrastructure without needing to rewrite the application.
You can ease in packages to Habitat, and then have Habitat manage pieces of the application.
Habitat is Cloud-agnostic and integrates with public and private Cloud providers by exporting an application as a container.
Chef is one of just a few third-party offerings marketed directly by AWS.
From inception to deployment, there is a place for large Cloud providers to parlay into language they already speak.
Operation Code is a non-profit that teaches software engineer skills to veterans. It helps veterans transition into high-paying engineering jobs.
The technology landscape is ever changing. What skills are most marketable?
Operation Code is a learning by experience type of organization and usually starts people on the front-end to immediately see results.

Links:

Nell Shamrell-Harrington

Nell Shamrell-Harrington on Twitter

Nell Shamrell-Harrington on GitHub

GorillaStack (use discount code: screaming)

Episode 4: It's a Data Lake, not a Data Public Swimming Pool

Corey Quinn — Wed, 04 Apr 2018 06:00:00 +0000

Open source activism tends to focus on running on hardware you can trust and avoiding Cloud computing. The problem with some Cloud providers has to do with a conflict of interest between serving customers and how they generate revenue. It’s important for the customer to have control of their computer and their data in the Cloud. But what about their security and privacy?

Today, we’re talking to Kyle Rankin, chief security officer at Purism and writer for Linux Journal. He is a Linux expert who decided to work at Purism because of the company’s belief in free software and the Linux community.

Some of the highlights of the show include:

Cloud providers have faced challenges when it comes to data privacy and who owns what.
The word “Cloud” is overloaded, and it is unclear who is in control.
Cloud providers can sabotage efforts to make programs work together.
Cloud providers may not troll through data and exploit it. Yet, they develop tools for customers to be able to do that.
Even though Linux Journal stopped being printed and went digital, and was going under, it’s now back and taking a new approach.
What matters to new readers and Linux users is now different than what was important to original readers.
The more time you can spend to understand what’s happening behind the scenes will make you much more marketable and adaptable.
Kyle explains whether Amazon Linux is becoming a viable concern and if distribution matters anymore. Now, it’s about running an application, not thinking about what it’s running on.
Are there gangs of Cloud users? Do people look down on Azure users? The target is always moving and changing.
Check out Kyle’s book, Linux Hardening in Hostile Networks: Server Security from TLS to Tor.

Links:

Kyle Rankin on Twitter
Purism
Kyle Rankin’s book - Linux Hardening in Hostile Networks: Server Security from TLS to Tor
Linux Journal 2.0 FAQ
GorillaStack (use “screaming” for discount)

Episode 3: Turning Off Someone Else's Site as a Service

Corey Quinn — Wed, 28 Mar 2018 00:00:00 +0000

How do you encourage businesses to pick Google Cloud over Amazon and other providers? How do you advocate for selecting Google Cloud to be successful on that platform? Google Cloud is not just a toy with fun features, but is a a capable Cloud service.

Today, we’re talking to Seth Vargo, a Senior Staff Developer Advocate at Google. Previously, he worked at HashiCorp in a similar advocacy role and worked very closely with Terraform, Vault, Consul, Nomad, and other tools. He left HashiCorp to join Google Cloud and talk about those tools and his experiences with Chef and Puppet, as well as communities surrounding them. He wants to share with you how to use these tools to integrate with Google Cloud and help drive product direction.

Some of the highlights of the show include:

Strengths related to Google Cloud include its billing aspect. You can work on Cloud bills and terminate all billable resources. The button you click in the user interface to disable billing across an entire project and delete all billable resources has an API. You can build a chat bot or script, too. It presents anything you’ve done in the Consul by clicking and pointing, as well as gives you what that looks like in code form.
You can expose that from other people’s accounts because turning off someone else’s Website as a service can be beneficial. You can invite anyone with a Google account, not just ‘@gmail.com’ but ‘@’ any domain and give them admin or editor permissions across a project. They’re effectively part of your organization within the scope of that project. For example, this feature is useful for training or if a consultant needs to see all of your different clients in one dashboard, but your clients can’t see each other.
Google is a household name. However, it’s important to recognize that advocacy is not just external advocacy, there’s an internal component to it. There’s many parts of Google and many features of Google Cloud that people aren’t aware of. As an advocate, Seth’s job is to help people win.
Besides showing people how they can be successful on Google Cloud, Seth focuses on strategic complaining. He is deeply ingrained in several DevOps and configuration management communities, which provide him with positive and negative feedback. It’s his job to take that feedback and convert it into meaningful action items for product teams to prioritize and put on roadmaps. Then, the voice of the communities are echoed in the features and products being internally developed.
Amazon has been in the Cloud business for a long time. What took Google so long? For a long time, Google was perceived as being late to the party and not able to offer as comprehensive and experienced services as Amazon. Now, people view Google Cloud as not being substandard, but not where serious business happens. It’s a fully feature platform and it comes down to preferences and pre-existing features, not capability.
Small and mid-size companies typically pick a Cloud provider and stick with their choice. Larger companies and enterprises, such as Fortune 50 and Fortune 500 companies, pick multiple Clouds. This is usually due to some type of legal compliance issues, or there are Cloud providers that have specific features.
Externally at Google, there is the Deployment Manager tool at cloud.google.com. It’s the equivalent of CloudFormation, and teams at Google are staffed full time to perform engineering work on it. Every API that you get by clicking a button on cloud.google.com are viewing the API Docs accessible via the Deployment Manager.
Google Cloud also partners with open source tools and corresponding companies. There are people at Google who are paid by Google who work full time on open source tools, like Terraform, Chef, and Puppet. This allows you to provision Google Cloud resources using the tools that you prefer.
According to Seth, there’s five key pillars of DevOps: 1) Reduce organizational silos and break down barriers between teams; 2) Accept failures; 3) Implement gradual change; 4) Tooling and automation; and 5) Measure everything.
Think of DevOps as an interface in programming language, like Java, or a type of language where it doesn’t actually define what you do, but gives you a high level of what the function is supposed to implement.
With the SRE discipline, there’s a prescribed way for performing those five pillars of DevOps. Specific tools and technologies used within Google, some of which are exposed publicly as part of Google Cloud, enable the kind of DevOps culture and DevOps mindset that occur.
A reason why Google offers abstract classes in programming is that there’s more than one way to solve a problem, and SRE is just one of those ways. It’s the way that has worked best for Google, and it has worked best for a number of customers that Google is working with. But there are some other ways, too. Google supports those ways and recognizes that there isn’t just one path to operational success, but many ways to reach that prosperity.
The book, Site Reliability Engineering, describes how Google does SRE, which tried to be evangelized with the world because it can help people improve operations. The flip side of that is that organizations need to be cognizant of their own requirements.
Google has always held up along several other companies as a shining beacon of how infrastructure management could be. But some say there’s still problems with its infrastructure, even after 20-some years and billions invested.
Every company has problems, some of them technical, some cultural. Google is no exception. The one key difference is the way Google handles issues from a cultural perspective. It focuses on fixing the problem and making sure it doesn’t happen again. There’s a very blameless culture.
Conferences tend to include a lot of hand waving and storytelling. But as an industry, more war stories need to be told instead of pleasure stories. Conference organizers want to see sunshine and rainbows because that sells tickets and makes people happy. The systemic problem is how to talk about problems out in the open.
Becoming frustrated and trying to figure out why computers do certain things is a key component of the SRE discipline referred to as Toil - work tied to systems that either we don’t understand or don’t make sense to automate.
Those going to Google Cloud to ‘move and improve’ tend to be a mix of those from other Cloud providers and those from on-premise data center deployments. Move and improve is where there are VMs in a data center, and they need to be moved to the Cloud.
There are tiny differences around the Cloud-native paradigm and providers. There’s some key pillars: Does it handle restarts well? Is it highly available? Can it be containerized, even though containers aren’t necessarily required for Cloud native? Does it package all of its dependencies with it? Can it run on different operating systems? All of these things are generic, they’re not specific to a Cloud provider.

Links:

Google Cloud and blog

Site Reliability Engineering book for O’Reilly

Quotes by Seth:

“Everything we do on Google Cloud is API First. Anytime you click a button in that Web UI, there is a corresponding API call, which means you can build automation, compliance, and testing around these various aspects.”

“The IAM and permission management in Google Cloud is incredibly powerful. It leverages the same IAM permissions that G Suite has which is hosted Gmail, Calendar, and all of those other things.”

“How do I get people who want to use Google Cloud or don’t know about Google Cloud? The ability to be successful on the platform.”

“I would definitely say that any company you work at, whether the recruiter tells you that it’s all sunshine and rainbows and there’s nothing ever wrong is a lie.”

Episode 2: Shoving a SAN into us-east-1

Corey Quinn — Wed, 21 Mar 2018 00:00:00 +0000

When companies migrate to the Cloud, they are literally changing how they do everything in their IT department. If lots of customers exclusively rely on a service, like us-east-1, then they are directly impacted by outages. There is safety in a herd and in numbers because everybody sits there, down and out. But, you don’t engineer your application to be a little more less than a single point of failure. It’s a bad idea to use a sole backing service for something, and it’s unacceptable from a business perspective.

Today, we’re talking to Chris Short from the Cloud and DevOps space. Recently, he was recognized for his DevOps’ish newsletter and won the Opensource.com People’s Choice Award for his DevOps writing. He’s been blogging for years and writing about things that he does every day, such as tutorials, codes, and methods. Now, Chris, along with Jason Hibbets, run the DevOps team for Opensource.com

Some of the highlights of the show include:

Chris’ writing makes difficult topics understandable. He is frank and provides broad information. However, he admits when he is not sure about something.
SJ Technologies aims to help companies embrace a DevOps philosophy, while adapting their operations to a Cloud-native world. Companies want to take advantage of philosophies and tooling around being Cloud native.
Many companies consider a Cloud migration because they’ve got data centers across the globe. It’s active-passive backup with two data centers that are treated differently and cannot switch to easily.
Some companies do a Cloud migration to refactor and save money. A Cloud migration can result in you having to shove your SAN into the USC1. It can become a hybrid workflow.
Lift and shift is often considered the first legitimate step toward moving to the Cloud. However, know as much as you can about your applications and RAM and CPU allowances. Look at density when you’re lifting and shifting.
Know how your applications work and work together. Simplify a migration by knowing what size and instances to use and what monitoring to have in place.
Some do not support being on the Cloud due to a lack of understanding of business practices and how they are applied. But, most are no longer skeptical about moving to the Cloud. Now, instead of ‘why cloud,’ it becomes ‘why not.’

Don’t jump without looking. Planning phases are important, but there will be unknowns that you will have to face.
Downtime does cost money. Customers will go to other sites. They can find what they want and need somewhere else. There’s no longer a sole source of anything.
The DevOps journey is never finished, and you’re never done migrating. Embrace changes yourself to help organizations change.

Links:

Chris Short on Twitter

DevOps'ish

SJ Technologies

Amazon Web Services

Cloud Native Infrastructure

The Mythical Man-Month

Atlassian

BuzzFeed

Quotes by Chris:

“Let’s not say that they’re going whole hog Cloud Native or whole hog cloud for that matter but they wanna utilize some things.”

“They can never switch from one to the other very easily, but they want to be able to do that in the Cloud and you end up biting off a lot more than you can chew…”

“Create them in AWS. Go. They gladly slurp in all your VM where instances you can create a mapping of this sized thing to that sized thing and off you go. But it’s a good strategy to just get there.”

“We have to get better as technologists in making changes and helping people embrace change.”

Episode 1: Feature Flags with Heidi Waterhouse of LaunchDarkly

Corey Quinn — Mon, 19 Mar 2018 14:51:00 +0000

This podcast features people doing interesting work in the world of Cloud. What is the state of the technical world? Let’s first focus on the up or down, on or off function of feature flags.

Today, we’re talking to Heidi Waterhouse, a technical writer turned Developer Advocate at LaunchDarkly, which is a feature flag service - a way to wrap a snippet of code around your feature and make it into an instrument to turn on or off. It lets you turn things on and off in your codebase quickly without having to do several commits. However, it is difficult to track it when there are more than about a dozen flags. So, LaunchDarkly provides a way to manage your features at scale with a usable interface and API.

Some of the highlights of the show include:

A feature flag allows you to hide items before you want them to go live on your Website. You hide it behind a feature flag, doing all the work ahead of time. Then, at some point, you turn it all on instantly without the risk of pushing untested code into your production.
You can test at scale to gain authentic data. Test something with your team, your company’s employees, your customers, etc. However, no matter how good your integration tests are, there’s always wobbles to watch for in the system.
With implementation, there are a few paths that can work, such as the massive reorganization path. Or, you can just start incrementally with feature flags for new features.
LaunchDarkly thinks in the Cloud as the surface because it mostly works with people who are doing Web-based delivery of features.

Major companies, like Google and Facebook, offer services similar to feature flags for their own development. They’re operating on such a giant scale that they have internal teams doing it.
Companies use feature flags on the front-end and other purposes. It works through the whole stack from frontend page delivery, pricing tiers, white labeling, style sheets, to safer deployments.
Do not focus on documentation. You should not have to read documentation for anything that you don’t own. Every feature should have documentation tied to its code. Create a customized experience.
Feature flags effectively manage and minimize risk. There is always risk in the world, but what causes disaster is not just one failure. It is a multiplication of failures. This goes wrong and that goes wrong. Feature flagging breaks monolithic releases into tiny chunks that can go forward or backward.
LaunchDarkly holds monthly meet-ups called, Test and Production. People share their use case regarding continuous integration, continuous deployment, DevOps, etc.

Links:

Quotes by Heidi:

“What feature flags do is make it possible for you to push out a deployment with things hidden, we call it launching darkly.”

“We’re all about avoiding risk, I think this is our motto this year, eliminate risk…you can’t eliminate risk, but you can make it much less risky.”

“Go ahead and write your feature. You know that it’s hidden behind the magical feature flying curtain until you’re ready to turn it on.”

“If 20 years of technical writing taught me anything, it’s that nobody wants to be reading documentation.”