layout: true background-image: url(slides_files/RStudio-Logo-Flat.png) background-position: 2% 98% background-size: 10% --- name: title class: inverse, center, middle # **APIs**: you're probably not using them and why you probably should ### Presented by: **Katie Masiello** | Developed by: **Josiah Parry** ### **RStudio Customer Success** ??? I've been working around APIs for over a year and gleaning pieces from talks here and there, but it wasn't until recently when my colleague Josiah and I were talking through this material today that I felt like I'd internalized it and it all really clicked. I hope the discussion today is transformative for you as well. --- class: middle, center #Where are you in your data science journey? #Where is your organization? ??? I want to start by framing things a bit asking you to think about where you are in your data science journey? Are you brand new to this and the tools, are you in a place where you and/or the organization needs to mature and adopt new tools and workflows? I get to talk to a lot of teams in this role and what is neat is getting a pulse on where teams are on the continuum of maturity. --- # Common Pain Points * "I've created a Shiny beast!" * "How do we share this code with another system / group?" * "We are replicating work in every project" ??? Often when teams are going through some maturation pains, we hear things like this. Sometimes you build out a solution but the solution itself becomes the problem in that it's either difficult to maintain or update, or supporting it in production is unwieldy. These are some of the ways individuals and teams take steps in maturity. --- class: inverse, center, middle # Changing Data Science Landscape ??? organizations and the overall landscape for data science is maturing as well. --- # Data Science in 2021 * Part of production applications * Beyond ad hoc reports, visualizations, and dashboards * Need to equip data scientists with production tools * Dependence on IT * To succeed, we need to embrace DevOps * How? Microservices ??? we see modern data science teams doing these things. --- # Monolithic Mindset <img src="slides_files/monolith.png" width="60%" style="display: block; margin: auto;" /> ??? a solution may start out as what we think of as a monolith. A singular beast. but I'd like to encourage you to think of a more efficient way to build, productionize, and sustain assets. --- # Lean it out with Microservices <img src="slides_files/microservices.png" width="90%" style="display: block; margin: auto;" /> ??? we want to lean things out with what we call microservices. smaller assets that do singular work --- class: inverse, middle .left[###The Single Responsibility Principle] .center[# “Gather together those things that change for the same reason, and separate those things that change for different reasons.”] .right[###– Robert C. Martin] ??? If you follow best practices with version control / github, you'll undoubtedly be diligent about committing changes in small bundles all aligned with one particular piece of work. this is the same concept. --- class: inverse, center, middle # Okay, but how? --- class: inverse, center, middle # APIs, that's how. --- class: inverse, center, middle # AP-who? --- # TL;DR - APIs * Application Programming Interfaces * Machines talking to machines * RESTful APIs use HTTP * Think of HTTP as a universal language ??? these are the basics of APIs. What you may have heard before, right? It's how machines talk to one another. You're using an API just to call up a website like cnn. But lets get into it a bit more to help internalize this a bit more. --- # Think of APIs as a way to access functions General form of an API: ``` http://hostname/api_endpoint/?parameter=value ``` Example function: ```r my_function <- function(argument) { some_kind_of_operation(argument) } ``` As an API: ``` http://hostname/my_function/?argument=value ``` ??? you write a function in R to do something. an API becomes a way to access that. the general form is: and it maps out like this: --- # Think of APIs as a universal adapter .center[<img src="slides_files/dockingstation.png" height = 325>] .center[<img src="slides_files/dockingstationDescription.png" width = 1000> ] ??? I'm a mechanical engineer by training. I think in terms of physical things. tools. nuts and bolts and fasteners. fluid flowing in pipes. so another way to think of APIs is like an adapter. you've got this black box thing. it doesnt really matter what's inside, how it got there, what language it was written in, or anything else. As long as you've got something that fits into the USB-shaped hole that's in it, you can plug something in, like a keyboard or thumb drive into that hole. the input signal that you send to the black box adapter interfaces with the computer and you get some kind of output in response. this is equivalent to plugging your R or Python based code into the API's endpoint with specific input parameters and the API returns an output. --- # The Case for APIs Traditional Data Science Tool Box * Reports * Presentations * Spreadsheets * Interactive Dashboards * *all limited to end user consumption* Extend your reach with APIs! ??? So now we know more about what an api is. What's so great about them? For one, if we look at the traditional data science toolbox it may include any number of these traditional things. but they're all limited to end user consumption. they require some kind of user interaction. an API permits you to eliminate the human interaction. --- # The Case for APIs .pull-left[<h3 style="margin-top: 0; color: #447099;"> Organizational </h3> * Smaller pieces are easier to manage * Component monitoring * Infrastructure for recycled code ♻️ * Less technical debt! * Better change management * Upgrade one service at a time without bringing down whole application ] .pull-right[<h3 style="margin-top: 0; color: #447099;"> Collaborative </h3> * Less friction and effort in tool handoff * Empower teams to use preferred tool chain 🔥 * Concerned about API requests not software language * No need to translate work * Programmatic use of services * *Glue together API requests! Pipelines, baby!* <img src="slides_files/pipe.png" height="20" style="vertical-align:middle"> ] ??? this is a lot but a few things i want to point out - APIs let you lean things out like we talked about. breaking things into smaller, reusable chunks. you can spend less time troubleshooting and focus on the smaller pieces of the puzzle for change management and upgrading. Even more compelling I Think is you can eliminate friction between groups. teams can use whatever language is preferred and there's no recoding required. this enables automation and programmatic use of services, and you really feel like a superstar when you start gluing together pipelines of API requests. --- class: inverse name: Signs you might be ready to create an API .left-column-impact[ <h6 style="color:#FFFFFF"> 6</h6> <h7>signs you might be ready to create an API</h7> ] .right-column[ <br><br> ### Redundant code (copying and pasting/reusing) ### You have a large app that's hard to manage ### You have common processes / business logic that should be used ### Other teams want your code ### Other teams may not know your language ### You want to integrate different software into one toolchain ] ??? we've talked about these items already. but if you find this discussion today resonating with where you are today, it may be time to think about brining in APIs. --- class: inverse, center, middle # How do you build an API? --- # API frameworks * R - plumber * Python - Flask, Fast API * JavaScript - Node.js & Express.js --- class: inverse, center, middle # How do you deploy an API? --- # Deployment Options * RStudio Connect <br> <img src="slides_files/rsc.png" height="100" style="vertical-align:middle"> * Digital Ocean <br><img src="slides_files/digitalocean.png" width="100" style="vertical-align:middle"> * Docker <br><img src="slides_files/docker.png" width="100" style="vertical-align:middle"> --- class: inverse, center, middle # Let's get into it --- # Parts of an API * **Host (fixed):** * <span style="background-color:#fff2cc">`http://api.hostname.io/`</span> * **Endpoint:** * Resource location (think of as a function) * `http://api.hostname.io/`<span style="background-color:#fff2cc">`end-point`</span> * **Parameters (optional):** * Address varying parts of a request * `http://api.hostname.io/end-point/`<span style="background-color:#fff2cc">`?param=value`</span> * **Headers & body (optional):** * (not in URL) Associated (meta)data ??? we've used these words already but lets reiterate some vocabulary and concepts. --- # API Requests * Each API endpoint has a different **method** * GET * Used to <span style="background-color:#fff2cc">*retrieve*</span> data. Parameters only. No body. * Everything is in the URL. * Don’t send sensitive data! * POST * Used for sending data (files or text). More secure. * Creating or modifying something. * Other methods: * PUT * DELETE (yikes!) ☠️ * HEAD * … --- # Example: Go/No-Go Model ![:col_header <img src="slides_files/info-sec.png" height="100" style="vertical-align:middle";>, <img src="slides_files/statistician.png" height="100" style="vertical-align:middle";>, <img src="slides_files/jr-analyst.png" height="100" style="vertical-align:middle"; >] ![:col_row Susan has built a tool in R that provides Go/No-Go assertions, Andrea needs to incorporate these assertions into her Python code to inform her model, Mateo needs his daily metrics shown in Tableau to reflect these assertions] -- ```r #Go/No-Go script sample(c("heads", "tails"), 1, prob = c(0.5, 0.5), replace = TRUE) ``` ``` ## [1] "tails" ``` -- Yeah, it's a coin toss. Let's keep it simple. 😄 --- class: inverse .left-column-impact[ <h6 style="color:#FFFFFF">4</h6> <h7>steps to making a plumber API</h7> ] .right-column[ <br><img src="slides_files/plumber.png" style="vertical-align:middle" height = 150><br> ###1. Make a function ###2. Add `plumber` decorations ###3. ⚠️ Save as **"`plumber.r`"** ###4. Publish ] --- # Make it functional .panelset[ .panel[.panel-name[Function basics] * Functions are code shortcuts! * Arguments are placeholders * Things that change should be arguments * Whatever is printed last is returned .pull-left[**Tip 1: use code snippets to quickly insert a function** .center[<img src="slides_files/function_snippet.gif" style="vertical-align:middle" height = 150>]] .pull-right[**Tip 2: Extract function to convert code to a function** .center[<img src="slides_files/extract_function.gif" style="vertical-align:middle" height = 150>]] ] .panel[.panel-name[Coin Toss Function] <code class ='r hljs remark-code'>sample(c("heads", "tails"), 1, <br> prob = c(0.5, 0.5), replace = TRUE)</code> ``` ## [1] "tails" ``` <code class ='r hljs remark-code'>#Function 1: toss once and give me a heads or tails output <br>coin_toss <- <span style='background-color:#ffff7f'>function</span>() {<br> sample(c("heads", "tails"), 1, <br> prob = c(0.5, 0.5), replace = TRUE)<br>}<br>coin_toss()</code> ``` ## [1] "tails" ``` ] .panel[.panel-name[Multi-Toss Function] <code class ='r hljs remark-code'>sample(c("heads", "tails"), <span style='background-color:#ffff7f'>1</span>, <br> prob = c(0.5, 0.5), replace = TRUE)</code> ``` ## [1] "heads" ``` <code class ='r hljs remark-code'>#Function 2: toss multiple times and summarize output<br>multi_toss <- function(<span style='background-color:#ffff7f'>n</span>) {<br> table(sample(c("heads", "tails"), <span style='background-color:#ffff7f'>n</span>, <br> prob = c(0.5, 0.5), replace = TRUE)) %>% <br> as.data.frame()<br>}<br>multi_toss(<span style='background-color:#ffff7f'>500</span>)</code> ``` ## Var1 Freq ## 1 heads 249 ## 2 tails 251 ``` ] ] --- #Get ready to decorate! ```r #Function 1: toss once and give me a heads or tails output coin_toss <- function() { sample(c("heads", "tails"), 1, prob = c(0.5, 0.5), replace = TRUE) } #Function 2: toss multiple times and summarize output multi_toss <- function(n) { table(sample(c("heads", "tails"), n, prob = c(0.5, 0.5), replace = TRUE)) %>% as.data.frame() } ``` .center[<img src="slides_files/decorate.gif" style="vertical-align:middle" height = 150>] --- # The Finished API Note special plumber comments (decorations) with `#*` ```r library(plumber) #* @apiTitle Coin Toss API #* Toss a coin once and give the result #* @get /coin_toss function() { sample(c("heads", "tails"), 1, prob = c(0.5, 0.5), replace = TRUE)} #* Toss a coin multiple times and summarize output #* @param n the number of times to toss the coin #* @get /multi_toss function(n) { as.data.frame( table(sample(c("heads", "tails"), n, prob = c(0.5, 0.5), replace = TRUE)))} ``` --- # The API - API title ```r library(plumber) *#* @apiTitle Coin Toss API #* Toss a coin once and give the result #* @get /coin_toss function() { sample(c("heads", "tails"), 1, prob = c(0.5, 0.5), replace = TRUE) } #* Toss a coin multiple times and summarize output #* @param n the number of times to toss the coin #* @get /multi_toss function(n) { as.data.frame( table(sample(c("heads", "tails"), n, prob = c(0.5, 0.5), replace = TRUE))) } ``` --- # The API - Endpoint description ```r library(plumber) #* @apiTitle Coin Toss API *#* Toss a coin once and give the result #* @get /coin_toss function() { sample(c("heads", "tails"), 1, prob = c(0.5, 0.5), replace = TRUE) } *#* Toss a coin multiple times and summarize output #* @param n the number of times to toss the coin #* @get /multi_toss function(n) { as.data.frame( table(sample(c("heads", "tails"), n, prob = c(0.5, 0.5), replace = TRUE))) } ``` --- # The API - Endpoint parameters ```r library(plumber) #* @apiTitle Coin Toss API #* Toss a coin once and give the result #* @get /coin_toss function() { sample(c("heads", "tails"), 1, prob = c(0.5, 0.5), replace = TRUE) } #* Toss a coin multiple times and summarize output *#* @param n the number of times to toss the coin #* @get /multi_toss function(n) { as.data.frame( table(sample(c("heads", "tails"), n, prob = c(0.5, 0.5), replace = TRUE))) } ``` --- # The API - Endpoint method ```r library(plumber) #* @apiTitle Coin Toss API #* Toss a coin once and give the result *#* @get /coin_toss function() { sample(c("heads", "tails"), 1, prob = c(0.5, 0.5), replace = TRUE) } #* Toss a coin multiple times and summarize output #* @param n the number of times to toss the coin *#* @get /multi_toss function(n) { as.data.frame( table(sample(c("heads", "tails"), n, prob = c(0.5, 0.5), replace = TRUE))) } ``` --- class: inverse, center, middle # The Swagger Interface --- # Swagger <img src="slides_files/swagger.png" width="80%" style="display: block; margin: auto;" /> ??? The Swagger interface in the RStudio IDE and on RStudio Connect permits you to interact with the API --- # Publish the API Using RStudio Connect (push button deployment, git-backed deployment, or programmatic) .center[**<https://colorado.rstudio.com/rsc/coin_api/>**] <img src="slides_files/published.png" width="80%" style="display: block; margin: auto;" /> --- class: inverse, middle, center #Hey, we published an API! ##Let's go play --- #Query our API from a browser Single Coin Toss: <https://colorado.rstudio.com/rsc/coin_api/coin_toss> <br><br> <img src="slides_files/coin_toss_result.png" style="vertical-align:middle"> Multi Coin Toss: <https://colorado.rstudio.com/rsc/coin_api/multi_toss?n=40> <br><br> <img src="slides_files/multi_toss_result.png" style="vertical-align:middle"> --- # Elements of our API call .center[ <h3><span style='color:tomato'>colorado.rstudio.com/rsc/</span><span style='color:cornflowerblue'>coin_api/</span><span style='color:orange'>multi_toss</span><span style='color:darkseagreen'>?n=40</span></h3> ### <span style="color:tomato">host/</span><span style="color:cornflowerblue">RSC content path/</span><span style="color:orange">endpoint</span><span style="color:darkseagreen">?parameters</span> ] `#* Toss a coin multiple times and summarize output` <span style="color:darkseagreen">`#* @param n the number of times to toss the coin`</span> <span style="color:orange">`#* @get /multi_toss`</span> ``` function(n) { as.data.frame( table(sample(c("heads", "tails"), n, prob = c(0.5, 0.5), replace = TRUE))) } ``` ??? So to reiterate a few things - remember that in our API call for multitoss we have the following pieces. and here's how they map back to the pariticular section in the plumber.r API script. --- class: inverse, middle, center # That's great, now how do I use it? --- class: inverse, middle, center # Calling the API from R {httr} --- .left-column-impact[ <h7 style="color:#447099"><br><br>API Call Steps</h7> ] .right-column[ <br><br> * ###Generate the query URL * ###Make the request * ###Parse the response * ###(This pattern is the same for all languages) ] --- # Creating the URL 1. Specify the base URL 2. Create named list with parameters and values 3. Use `modify_url()` to fill in parameters ```r library(httr) base_url <- "https://colorado.rstudio.com/rsc/coin_api/" endpoint <- "multi_toss" params <- list(n = 1000) *query_url <- modify_url(url = paste0(base_url, endpoint), * query = params) ``` `[1] "https://colorado.rstudio.com/rsc/coin_api/multi_toss?n=1000"` --- # Send the request * Use the appropriate method * e.g. `GET()`, `POST()`, `PUT()` <span style="background-color:#fff2cc">`response <- GET(query_url)`</span> ``` > GET(query_url) Response [https://colorado.rstudio.com/rsc/coin_api/coin_toss] Date: 2021-01-08 18:56 Status: 200 Content-Type: application/json Size: 9 B ``` 🤔 --- # Parse the request ```r *(response_raw <- content(response, as = "text")) ## [1] "[{\"Var1\":\"heads\",\"Freq\":500},{\"Var1\":\"tails\",\"Freq\":500}]" *jsonlite::fromJSON(response_raw) ## Var1 Freq ## 1 heads 515 ## 2 tails 485 ``` Putting everything together: ```r (response <- GET(query_url) %>% content(as = "text") %>% jsonlite::fromJSON()) ## Var1 Freq ## 1 heads 511 ## 2 tails 489 ``` --- class: inverse # What have we done? .left-column-impact[] .right-column[ ### <svg style="height:0.8em;top:.04em;position:relative;fill:white;" viewBox="0 0 448 512"><path d="M400 480H48c-26.51 0-48-21.49-48-48V80c0-26.51 21.49-48 48-48h352c26.51 0 48 21.49 48 48v352c0 26.51-21.49 48-48 48zm-204.686-98.059l184-184c6.248-6.248 6.248-16.379 0-22.627l-22.627-22.627c-6.248-6.248-16.379-6.249-22.628 0L184 302.745l-70.059-70.059c-6.248-6.248-16.379-6.248-22.628 0l-22.627 22.627c-6.248 6.248-6.248 16.379 0 22.627l104 104c6.249 6.25 16.379 6.25 22.628.001z"/></svg> Turn our R code into an API ### <svg style="height:0.8em;top:.04em;position:relative;fill:white;" viewBox="0 0 448 512"><path d="M400 480H48c-26.51 0-48-21.49-48-48V80c0-26.51 21.49-48 48-48h352c26.51 0 48 21.49 48 48v352c0 26.51-21.49 48-48 48zm-204.686-98.059l184-184c6.248-6.248 6.248-16.379 0-22.627l-22.627-22.627c-6.248-6.248-16.379-6.249-22.628 0L184 302.745l-70.059-70.059c-6.248-6.248-16.379-6.248-22.628 0l-22.627 22.627c-6.248 6.248-6.248 16.379 0 22.627l104 104c6.249 6.25 16.379 6.25 22.628.001z"/></svg> Publish our API ### <svg style="height:0.8em;top:.04em;position:relative;fill:white;" viewBox="0 0 448 512"><path d="M400 480H48c-26.51 0-48-21.49-48-48V80c0-26.51 21.49-48 48-48h352c26.51 0 48 21.49 48 48v352c0 26.51-21.49 48-48 48zm-204.686-98.059l184-184c6.248-6.248 6.248-16.379 0-22.627l-22.627-22.627c-6.248-6.248-16.379-6.249-22.628 0L184 302.745l-70.059-70.059c-6.248-6.248-16.379-6.248-22.628 0l-22.627 22.627c-6.248 6.248-6.248 16.379 0 22.627l104 104c6.249 6.25 16.379 6.25 22.628.001z"/></svg> Call a published API for use in R ] # 🎉 --- class: inverse, middle, center # Now What? --- # Time to Let Loose with APIs .left-column-impact[<img src="slides_files/lets_do_this.gif" style="vertical-align:middle">] .right-column[Brainstorm: API Ideas * Query your model or database via Slack integration * Standardize your data munging and other business logic * Pull the heavy lifting out of your Shiny app and into an API * *What use cases are you thinking?* Take it to the next level * Parallel execution - [Execute API calls asynchronously](!https://www.rstudio.com/resources/rstudioglobal-2021/plumber-and-future-async-web-apis/) * Mix up your serializer - beyond JSON, includes feather, csv, image types, [and more](!https://www.rplumber.io/reference/parsers.html) * Even more new features in the [plumber 1.1.0 release](!https://blog.rstudio.com/2021/03/29/plumber-v1-1-0/) ] --- class: inverse, middle, center ## Thank you! # Let's talk about it.