Lecture 20 Web APIs and Components

What is it?

A computing interface to a software component or a system, that defines how other components or systems can use it.

API vs. Application

Application/Library/Service
A blob of code that does things
API:
The way your code can ask the application/library/service to do things
The boundary of the application/library through which requests go in and responses come out

APIs vs. UIs

The Browser

API Aspects

Data model
what data does the API manipulate?
Methods
what can you ask it to do?
function names, arguments, return types
Syntax
In what language/format are the method invocations requested?

90s Web: HTTP

HTTP

  • API for talking to Web Servers
  • 6 main methods, pretty much as originally spec'ed by Tim Berners Lee
OPTIONS
“tell me about yourself”
GET
retrieve a resource
POST
upload a new resource
PUT
replace an existing resource
PATCH
modify a resource
DELETE
remove a resource

Method Parameters

Request

  • URL: “this” for method invocation
    • "object" you ask to do things
  • Body: content to PUT/POST/PATCH
  • Headers
    • Referer: page making the request
    • Accept: acceptable response data types (text, pdf, etc.)
    • Cookie: contains specific user data

Response

  • Status code: 200 OK, 301 redirect, 401 Unauthorized, 500 server error…
  • Body: content such as web page
  • Headers:
    • Content-Type: of returned object (text, pdf, etc.)
    • Set-Cookie: user data to store in browser
    • Location: Instruction to look elsewhere

Serialization

  • Network carries a stream of bytes/text
  • Each HTTP request is serialized into a big string with specific syntax
  • Web server receives and deserializes the string back into a structured request
  • Server then serializes response that returns to client
  • Which deserializes it into response data

							POST /echo?urlarg=newUser HTTP/1.1
							Host: scooterlabs.com
							Connection: keep-alive
							Content-Length: 39
							Cache-Control: max-age=0
							Upgrade-Insecure-Requests: 1
							Origin: http://localhost
							Content-Type: application/x-www-form-urlencoded
							User-Agent: Mozilla/5.0 (Windows NT 10.0……
							Accept: text/html,image/webp,img/apng…
							Referer: http://localhost/lectures/apis/form.html
							Accept-Encoding: gzip, deflate
							Accept-Language: en-US,en;q=0.9,he;q=0.8
					

HTTP Echo servers

Scooter labs runs a reflector service that will show you the arguments in any http request you make to it
  • Send back whatever you submitted
  • Good for testing your code
  • GET and POST Examples:
    
    									<a href="http://…/echo?myname=David…">
    
    									<form method="post"
    									  action="http://…/echo?urlArg=newUser">
    									  Input: <input name="textbox" type="text">

Idempotent and Safe Methods

The Ugly Reality

  • Developers are lazy
  • Can't be bothered with fine semantic distinctions of methods
  • Think only 2 really matter
  • Implement all actions through these

GET

  • arguments in query string (?) of url
  • easy to serialize whole request into url string
  • great for a few short method parameters

POST

  • body can contain serialized arguments
  • need work to build the body correctly
  • necessary for many/big arguments or uploads

00s Web: AJAX & REST

Origins

The Popular Approach

  • 90s web applications ran on the server
  • To take an action, user submitted a form
  • To await changes, keep reloading the page
  • Browser just displayed results and accepted user input

The Insight

  • Many actions change little of the UI
  • Client can do significant computation
  • Why bother involving the server?
    • Demands more server horsepower
    • Network makes slow UI
    • Reload loses your place in the page—disruptive
“The classic web application model works like this: Most user actions in the interface trigger an HTTP request back to a web server. The server does some processing — retrieving data, crunching numbers, talking to various legacy systems — and then returns an HTML page to the client. It’s a model adapted from the Web’s original use as a hypertext medium, but as fans of The Elements of User Experience know, what makes the Web good for hypertext doesn’t necessarily make it good for software applications.”

AJAX

Asynchronous
promises etc.
Javascript
computation on client
and XML
data serialization format
now replaced by JSON
but AJAJ doesn't sound as cool

Web APIs

Human vs. Machine

Web

HTML

https://github.com/karger


								<!DOCTYPE html>
								<html lang="en">
								<head>
								<meta charset="utf-8">
								<link rel="dns-prefetch" href="https://github.githubassets.com">
								<link rel="dns-prefetch" href="https://github-cloud.s3.amazonaws.com">
								<link rel="dns-prefetch" href="https://user-images.githubusercontent.com/">

								<link crossorigin="anonymous" media="all" integrity="sha512-bqIvTjrYM8hmo8Y7y5fzsxn+OAFrHWG6byMAbBSM9qFe8NQymQqFeBaF2MGvAwy7x2s4bnzXE5bDGHnrQjtoQQ==" rel="stylesheet" href="https://github.githubassets.com/assets/github-6ea22f4e3ad833c866a3c63bcb97f3b3.css" />

								<meta name="viewport" content="width=device-width">

								<title>karger (David Karger)</title>
								<meta name="description" content="karger has 31 repositories available. Follow their code on GitHub.">
								<link rel="search" type="application/opensearchdescription+xml" href="/opensearch.xml" title="GitHub">
								<link rel="fluid-icon" href="https://github.com/fluidicon.png" title="GitHub">
								<meta property="fb:app_id" content="1401488693436528">

								<meta name="twitter:image:src" content="https://avatars3.githubusercontent.com/u/1141086?s=400&v=4" /><meta name="twitter:site" content="@github" /><meta name="twitter:card" content="summary" /><meta name="twitter:title" content="karger - Overview" /><meta name="twitter:description"
								content="karger has 31 repositories available. Follow their code on GitHub." />

								<div class="js-profile-editable-area">
								<div class="hide-sm hide-md">
								<button name="button" type="button" class="btn btn-block mt-2 mb-3 js-profile-editable-edit-button" data-hydro-click="{"event_type":"user_profile.click","payload":{"profile_user_id":1141086,"target":"INLINE_EDIT_BUTTON","user_id":1141086,"originating_url":"https://github.com/karger"}}" data-hydro-click-hmac="15cf48f81af651c181ded3e3432629aaaed4c49127d0ccb02f75963a43bf695d">Edit profile</button>
								</div>

								<div class="p-note user-profile-bio mb-2 js-user-profile-bio"><div>
								Professor of Computer Science at MIT CSAIL, focusing on human computer interaction, end user application development, and computer supported collaborative work.</div></div>

								<ul class="vcard-details mb-3">
								<li itemprop="worksFor" show_title="false" aria-label="Organization: MIT" class="vcard-detail pt-1 css-truncate css-truncate-target"><svg class="octicon octicon-organization" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M16 12.999c0 .439-.45 1-1 1H7.995c-.539 0-.994-.447-.995-.999H1c-.54 0-1-.561-1-1 0-2.634 3-4 3-4s.229-.409 0-1c-.841-.621-1.058-.59-1-3 .058-2.419 1.367-3 2.5-3s2.442.58 2.5 3c.058 2.41-.159 2.379-1 3-.229.59 0 1 0 1s1.549.711 2.42 2.088C9.196 9.369 10 8.999 10 8.999s.229-.409 0-1c-.841-.62-1.058-.59-1-3 .058-2.419 1.367-3 2.5-3s2.437.581 2.495 3c.059 2.41-.158 2.38-1 3-.229.59 0 1 0 1s3.005 1.366 3.005 4z"></path></svg>
								<span class="p-org"><div>MIT</div></span>
								</li>
								<li itemprop="homeLocation" show_title="false" aria-label="Home location: Cambridge, MA" class="vcard-detail pt-1 css-truncate css-truncate-target"><svg class="octicon octicon-location" viewBox="0 0 12 16" version="1.1" width="12" height="16" aria-hidden="true"><path fill-rule="evenodd" d="M6 0C2.69 0 0 2.5 0 5.5 0 10.02 6 16 6 16s6-5.98 6-10.5C12 2.5 9.31 0 6 0zm0 14.55C4.14 12.52 1 8.44 1 5.5 1 3.02 3.25 1 6 1c1.34 0 2.61.48 3.56 1.36.92.86 1.44 1.97 1.44 3.14 0 2.94-3.14 7.02-5 9.05zM8 5.5c0 1.11-.89 2-2 2-1.11 0-2-.89-2-2 0-1.11.89-2 2-2 1.11 0 2 .89 2 2z"></path></svg>
								<span class="p-label">Cambridge, MA</span>
								</li>
					

API

https://api.github.com
/users/karger


							{
							"login": "karger",
							"id": 1141086,
							"node_id": "MDQ6VXNlcjExNDEwODY=",
							"avatar_url": "https://avatars2.githubusercontent.com/u/1141086?v=4",
							"gravatar_id": "",
							"url": "https://api.github.com/users/karger",
							"html_url": "https://github.com/karger",
							"followers_url": "https://api.github.com/users/karger/followers",
							"following_url": "https://api.github.com/users/karger/following{/other_user}",
							"gists_url": "https://api.github.com/users/karger/gists{/gist_id}",
							"starred_url": "https://api.github.com/users/karger/starred{/owner}{/repo}",
							"subscriptions_url": "https://api.github.com/users/karger/subscriptions",
							"organizations_url": "https://api.github.com/users/karger/orgs",
							"repos_url": "https://api.github.com/users/karger/repos",
							"events_url": "https://api.github.com/users/karger/events{/privacy}",
							"received_events_url": "https://api.github.com/users/karger/received_events",
							"type": "User",
							"site_admin": false,
							"name": "David Karger",
							"company": "MIT",
							"blog": "http://people.csail.mit.edu/karger",
							"location": "Cambridge, MA",
							"email": null,
							"hireable": null,
							"bio": "Professor of Computer Science at MIT CSAIL, focusing on human computer interaction, end user application development, and computer supported collaborative work.",
							"public_repos": 29,
							"public_gists": 1,
							"followers": 100,
							"following": 0,
							"created_at": "2011-10-20T15:41:03Z",
							"updated_at": "2020-04-24T23:02:34Z"
							}
					

Application-Specific APIs

Encapsulation

Wrapper Libraries

RESTful APIs

If you want to go much deeper into the weeds, here's an interesting article about how REST isn't really being used the way it was intended in that thesis.

The Problem

  • In many traditional services, server remembers state of session with client
    • who the user is
    • how far they've gotten in the activity
    • what comes next
  • Lots of this information is ephemeral
    • irrelevant once user closes the page
  • Significant burden on server to maintain
    • server doesn't know user has left
    • must keep state forever for every user who ever visited

The solution

  • REpresentational State Transfer
  • Keep all ephemeral state in the client
  • Every method call must include all the state the server needs to provide the method
  • Server still holds persistent state
    • your file uploads
    • your purchases
  • described in Roy Fielding's dissertation:

'10s: Security & Authorization

Why?

On Clients?

Authorization with Tokens

Web Token Types (I)

In URL parameters
  • http://site/?user=david&pwd=foo
  • terrible idea in general. Why?
    • URLs get saved, copied, shared
    • So tokens leak
    • But ok for new-user invite, password reset
    • Because url can be obsoleted quickly
Cookies
  • Associated with a particular domain
  • Server sends Set-Cookie: header in response from that domain
  • Client sends Cookie: header with (every) request to that domain
  • Consisten with REST:
    • client holds state, always sends to server
  • But remember client is not trusted

Web Token Types (II)

  • HTTP Basic Auth header
    • Authorization: Basic user:password
  • HTTP Digest Auth header
    • Authorization: Digest hash(user:passwd:nonce)
    • hashes username and pwd with nonce for security
  • HTTP Bearer Auth header
    • token is not username/password
    • instead, arbitrary string server can recognize
    • Authorization: bearer r8yc3n8t3ct9
    • decouples identity from authorization
    • server can know you're allowed without knowing who you are

Browser Built-in Handling

  • URL parameters
    • Browser can fetch url with included parameters
  • Cookie
    • Browser stores any cookie sent from server
    • sends cookie to server with every request
  • Bearer authentication
    • Must write custom js code to handle
  • Basic and Digest Auth
    • Server responds with challenge:
      
      									HTTP/1.1 401 Unauthorized
      									Date: Wed, 21 Oct 2015 07:28:00 GMT
      									WWW-Authenticate: Basic realm="asgard"
      							
    • Browser opens user/passwd dialog
    • and puts answer in appropriate Authentication: header
    • cached so user needn't re-type
    • Try it here

Handling in JS Code

  • URL
    • fetch()
  • Cookie
    • document.cookie property to set/get cookie
    • browser stores, sends with every request
  • Basic/Digest/Bearer Auth
    • Observe server challenge in response object
    • Send relevant Auth headers in fetch
      
      										fetch("https://httpbin.org/basic-auth/user/passwd", {
      										headers: new Headers({
      										"Authorization":
      										    `Basic ${base64.encode(`${user}:${pwd}`)}`
      										}),
      										}).then(response => {
      										if (!response.ok) throw new Error(response.status);
      										return response.json(); })
      								

Acquiring Tokens

API Keys

Security

Eavesdropper Attacks

Man In The Middle

Same Origin Policy

Some Attacks & Defenses

Stealing cookies/local storage
  • Attack:
    • Copy cookie from other page?
  • Defense:
    • Code like document.cookie only accesses cookie for domain of own page.
Iframes
  • Attack:
    • load target bank in iframe
    • iframe.contentDocument gives access to DOM inside iframe
    • inspect contents to steal data
    • submit withdrawal request
  • Defense:
    • Same origin policy means access to other-origin iframe.contentDocument returns null

Cross Site Scripting (XSS)

Attack

  • General attack: get your malicious script to run on (page from) target domain
  • Running in that domain, it has access to that domain's content
    • Can read/modify cookies
    • Can modify content on page to trick user
    • Can make API calls using domain's tokens

Example: Script Injection

  • Messaging client may display messages by appending to DOM with elt.innerHTML
  • Send a message:
    Hi Bob,<script src="http://my/my-attack.js"></script> how are you?
  • When message is appended, script runs in domain of app
  • Steal credentials of messaging client
  • Impersonate recipient

Cross Site Scripting

Example

  • Messaging client displays messages by appending to DOM with elt.innerHTML
  • Send a message:
    Hi Bob,<script src="http://my/my-attack.js"></script> how are you?
  • When message is appended, script runs in domain of app
  • Steal credentials of messaging client
  • Impersonate recipient

Defense

  • sanitize any content before you display it
  • make sure it isn't interpreted in an executable way
    • e.g., replace < character with &lt; entity
    • elt.textContent does this
  • same caution r.e. content you store on your server

What about script tags?

Question

  • A page can include <script> tags linking to other sites
  • This is important; it lets us link to useful remote libraries
  • JS can even dynamically inject such script tags in the page

Answer

  • Those scripts run in my domain
    • Cannot access other site!
  • As if I copied that code to my app
  • They're in my app because I put them there
  • Browser assumes I know what I'm doing
  • But note risk: malicious library can change its code later!

Cross-Site API Calls

  • Sometimes you want to pull data from other sites into your page
  • Or allow interaction with those sites, from your page
  • Iframe?
    • Style mismatch with your page
    • May want to combine that data with yours
    • May want to run that site's methods on your data
Embedded tweet pulling live data from Twitter

Should this be allowed?

No

  • If my company built an API, often I want only my site to have access
    • Don't want competitor sites stealing functionality of my API on their site!
    • Could require login to access, but
      • Requires users to setup accounts, which I might not want
      • Competitor could still steal my customers by using my API for them

Yes

  • I might want to share my API widely
    • Social good
    • Advertising/attractor for my site
  • I might want to share my API narrowly
    • Allow access by a few partner services

Cross Origin Resource Sharing (CORS)

OAuth

Motivation

  • Big hassle to have to log in to every site separately
  • Once the browser's knows I'm me, can't it prove it to every site where I have an account?
  • Skip all those per-site passwords, just prove I'm me once?
  • Some kind of globally accepted token?

Approach

  • Problem: if one token is globally accepted, everyone has it
    • So they can all pretend to be me?
  • Solution:
    • Distinct token per site
    • OAuth server generates tokens for me
    • I only need one permanent token
    • Use it to instruct OAuth server to create tokens for other sites

OAuth providers (servers)

OAuth providers (servers)

Summary: Web APIs

  • HTTP
    • 6 verbs GET/POST/PUT&hellp;
    • Headers carry method parameters
  • Web Application APIs
    • JSON object serialization
    • fetch() method
    • RESTFUL applications—all state on client


  • Please go/feedback
  • Authentication
    • Token to prove you are allowed
    • Might identify you or not
    • Different types
      • Params, Cookie, Basic, Digest, Bearer
    • OAuth offers centralization
  • Security
    • https vs. eavesdroping, man-in-middle
    • Same-Origin Policy protects sites
      • Script injection attacks try to hack
      • CORS offers controlled relaxation

Component-driven Development

Back in the day, websites often used raw buttons, lists, inputs, etc. for most of their design. But these days, large chunks of interfaces are composed of many pieces of text, images, symbols, and glyphs with complex styling applied to them.
Back in the day, websites often used raw buttons, lists, inputs, etc. for most of their design. But these days, large chunks of interfaces are composed of many pieces of text, images, symbols, and glyphs with complex styling applied to them.
Joshua Tree map
Turtle Rock

Turtle Rock

Joshua Tree National Park

California

About Turtle Rock

Descriptive text goes here.

<div class="location-card">
  <div class="card-map-header">
    <img src="map.jpg" alt="Joshua Tree map">
    <div class="card-photo-ring">
      <img class="card-photo"
           src="turtle-rock.jpg"
           alt="Turtle Rock">
    </div>
  </div>
  <div class="card-body">
    <div class="card-title-row">
      <div class="card-titles">
        <h2 class="card-name">Turtle Rock</h2>
        <p class="card-subtitle">
          Joshua Tree National Park
        </p>
      </div>
      <span class="card-region">California</span>
    </div>
    <hr class="card-divider">
    <section class="card-about">
      <h3>About Turtle Rock</h3>
      <p>Descriptive text goes here.</p>
    </section>
  </div>
</div>
<location-card
  name="Turtle Rock"
  subtitle="Joshua Tree National Park"
  region="California"
  map-src="map.jpg"
  photo-src="turtle-rock.jpg"
  description="Descriptive text goes here.">
</location-card>
When we start placing a lot of elements, the HTML can get very large. Ideally we'd like to encapsulate all of this HTML — and any associated logic — so we can easily reuse this card throughout our app without copy-pasting all those divs every time. With a component, we can collapse the whole thing into a single custom element that takes the data it needs as attributes.
In recent years, a new architecture is emerging: instead of separating by concerns, separating by self-contained, independent, reusable components. This approach was popularized by React starting around 2014–2015, though precursors exist in earlier UI frameworks and object-oriented programming. However, both can coexist: one can separate by component, and then separate concerns within each component.

Component File Structure


			my-app/
			├── index.html
			├── ButtonCounter/
			│   ├── ButtonCounter.js  ← JS logic + HTML template
			│   └── ButtonCounter.css
			└── ButtonCounterList/
			    ├── ButtonCounterList.js  ← JS logic + HTML template
			    └── ButtonCounterList.css
					
Here's what that separation of concerns looks like in practice. We make a separate folder for every component that has just the HTML, CSS, and JS we need to define it.

Vue.js Components

Component Definition

ButtonCounter.js


		import { ref } from "vue"

		export default {
		  setup() {
		    const count = ref(0)
		    return { count }
		  },
		  template: /*html*/`<button class="btn-counter" @click="count++">
		    Clicked {{ count }} times
		  </button>`
		}
		        
  • A component is a reusable Vue instance
  • setup() contains the component's logic
  • template is the HTML as a JS template string

ButtonCounter.css


		.btn-counter {
		  font-size: 1rem;
		  padding: 0.5em 1em;
		  border-radius: 6px;
		  border: 2px solid #2563eb;
		  background: #fff;
		  color: #2563eb;
		  cursor: pointer;
		}
		        

index.html


		<link rel="stylesheet" href="./ButtonCounter/ButtonCounter.css">
		<script type="module">
		import { createApp } from 'vue'
		import ButtonCounter from './ButtonCounter/ButtonCounter.js'

		createApp({
		  components: { ButtonCounter }
		}).mount('#app')
		</script>

		<div id="app">
		  <button-counter></button-counter>
		</div>
		        

Result

Vue components are plain JS objects with two main sections: - `setup()`: where reactive state and logic goes — return the values you want the template to access - `template`: the HTML for the component, as a template string We use a *default export* so the component can be imported by name from anywhere: `import ButtonCounter from './ButtonCounter/ButtonCounter.js'` To use the component in an app, register it in the `components` field, then mount the app on a DOM element. Once registered, Vue automatically converts the PascalCase name to a kebab-case tag, so `ButtonCounter` becomes `<button-counter>` It's usable just like a built-in HTML element!

Syntax Highlighting for Template Strings

Props

ButtonCounter.js


		import { ref } from "vue"

		export default {
		  props: ['initialCount'],
		  setup(props) {
		    const count = ref(props.initialCount ?? 0)
		    return { count }
		  },
		  template: /*html*/`<button class="btn-counter" @click="count++">
		    Clicked {{ count }} times
		  </button>`
		}
		        

index.html


		<div id="app">
		  <button-counter
		    initial-count="5"></button-counter>
		</div>
		        

Result

Props are how you create custom attributes for your component, just like `src` or `href` on a built-in HTML element. You declare which props a component accepts by passing an array of strings: each string is a prop name. Vue makes each prop available on the `props` object inside `setup()`, so you can read its value and wire it into your reactive state.

Passing Multiple Props

ButtonCounter.js


		import { ref } from "vue"

		export default {
		  props: ['initialCount', 'verb'],
		  setup(props) {
		    const count = ref(props.initialCount ?? 0)
		    return { count }
		  },
		  template: /*html*/`<button class="btn-counter" @click="count++">
		    {{ verb }} {{ count }} times
		  </button>`
		}
		        

index.html


		<div id="app">
		  <button-counter
		    initial-count="5"
		    verb="Pressed"></button-counter>
		</div>
		        

Result

We can pass multiple props by adding another string to the array. Notice that we can also use props directly in our template, not just in `setup()`!

Props Object Notation

ButtonCounter.js


		import { ref } from "vue"

		export default {
		  props: {
		    initialCount: { type: Number, default: 0 },
		    verb: { type: String, default: 'Clicked' }
		  },
		  setup(props) {
		    const count = ref(props.initialCount)
		    return { count }
		  },
		  template: /*html*/`<button class="btn-counter" @click="count++">
		    {{ verb }} {{ count }} times
		  </button>`
		}
		        

index.html


		<div id="app">
		  <button-counter
		    initial-count="5"
		    verb="Pressed"></button-counter>
		</div>
		        

Result

Instead of an array of strings, you can pass an object where each key is a prop name and the value is its configuration. This lets you specify the expected `type` (e.g. `Number`, `String`, `Boolean`) so Vue can warn you in the console if the wrong type is passed. You can also set a `default` value and mark a prop as `required: true`. The object form is preferred for any component that other people will use, since it acts as lightweight documentation.

Components Using Components

ButtonCounterList.js


		import { ref } from "vue"
		import ButtonCounter from '../ButtonCounter/ButtonCounter.js'

		export default {
		  components: { ButtonCounter },
		  setup() {
		    const items = ref([1, 2, 3])
		    return { items }
		  },
		  template: /*html*/`<ul>
		    <li v-for="n in items" :key="n">
		      <button-counter :initial-count="n"></button-counter>
		    </li>
		  </ul>`
		}
		        

Result

Similar to how we import components in our App, we can use components in other components. The syntax is very similar. We register the component in the `components` field and then we can use it in our template.