Ben Francis

tola.me.uk

What is a Web App?

2014-10-03 16:50 (UTC)

What is a web app? What is the difference between a web app and a web site? What is the difference between a web app and a non-web app?

In terms of User Experience there is a long continuum between “web site” and “web app” and the boundary between the two is not always clear. There are some characteristics that users perceive as being more “app like” and some as more “web like”.

The presence of web browser-like user interface elements like a URL bar and navigation controls are likely to make a user feel like they’re using a web site rather than an app for example, whereas content which appears to run independently of the browser feels more like an app. Apps are generally assumed to have at least limited functionality without an Internet connection and tend to have the concept of residing in a self-contained way on the local device after being “installed”, rather than being navigated to somewhere on the Internet.

From a technical point of view there is in fact usually very little difference between a web site and a web app. Different platforms currently deal with the concept of “web apps” in all sorts of different, incompatible ways, but very often the main difference between a web site and web app is simply the presence of an “app manifest”. The app manifest is a file containing a collection of metadata which is used when “installing” the app to create an icon on a homescreen or launcher.

At the moment pretty much every platform has its own proprietary app manifest format, but the W3C has the beginnings of a proposed specification for a standard “Manifest for web applications” which is starting to get traction with multiple browser vendors.

Web Manifest – Describing an App

Below is an example of a web app manifest following the proposed standard format.

http://example.com/myapp/manifest.json:

{
  "name": "My App",
  "icons": [{
    "src": "/myapp/icon.png",
    "sizes": "64x64",
    "type": "image/png"
  }],
  "start_url": "/myapp/index.html"
}

The manifest file is referenced inside the HTML of a web page using a link relation. This is cool because with this approach a web app doesn’t have to be distributed through a centrally controlled app store, it can be discovered and installed from any web page.

http://example.com/myapp/index.html:

<!DOCTYPE html>
<html>
  <head>
    <title>My App - Welcome</title>
    <link rel="manifest" href="manifest.json">
    <meta name="application-name" content="My App">
    <link rel="icon" sizes="64x64" href="icon.png">
...

As you can see from the example, these basic pieces of metadata which describe things like a name, an icon and a start URL are not that interesting in themselves because these things can already be expressed in HTML in a web standard way. But there are some other other proposed properties which could be much more interesting.

Display Modes – Breaking out of the Browser

We said above that one thing that makes a web app feel more app like is when it runs outside of the browser, without common browser UI elements like the URL bar and navigation controls. The proposed “display” property of the manifest allows authors of web content which is designed to function without the need for these UI elements to express that they want their content to run outside of the browser.

http://example.com/myapp/manifest.json:

{
  "name": "My App",
  "icons": [{
    "src": "/myapp/icon.png",
    "sizes": "64x64",
    "type": "image/png"
  }],
  "start_url": "/myapp/index.html"
  "scope": "/myapp"
  "display": "standalone"
}

The proposed display modes are “fullscreen”, “standalone”, “minimal-ui” and “browser”. The “browser” display mode opens the content in the user agent’s conventional method (e.g. a browser tab), but all of the other display modes open the content separate from the browser, with varying levels of browser UI.

There’s also a proposed “orientation” property which allows the content author to specify the default orientation (i.e. portrait/landscape) of their content.

App Scope – A Slice of the Web

In order for a web app to be treated separately from the rest of the web, we need to be able to define which parts of the web are part of the app, and which are not. The proposed “scope” property of the manifest defines the URL scope to which the manifest applies.

By default the scope of a web app is anything from the same origin as its manifest, but a single origin can also be sliced up into multiple apps or into app and non-app content.

Below is an example of a web app manifest with a defined scope.

http://example.com/myapp/manifest.json:

{
  "name": "My App",
  "icons": [{
    "src": "/myapp/icon.png",
    "sizes": "64x64",
    "type": "image/png"
  }],
  "start_url": "/myapp/index.html"
  "scope": "/myapp"
}

From the user’s point of view they can browse around the web, seamlessly navigating between web apps and web sites until they come across something they want to keep on their device and use often. They can then slice off that part of the web by “bookmarking” or “installing” it on their device to create an icon on their homescreen or launcher. From that point on, that slice of the web will be treated separately from the browser in its own “app”.

Without a defined scope, a web app is just a web page opened in a browser window which can then be navigated to any URL. If that window doesn’t have any browser-like navigation controls or a URL bar then the user can get stranded at a dead on the web with no way to go back, or worse still can be fooled into thinking that a web page they thought was part of a web app they trust is actually from another, malicious, origin.

The web browser is like a catch-all app for browsing all of the parts of the web which the user hasn’t sliced off to use as a standalone app. Once a web app is registered with the user agent as managing a defined slice of the web, the user can seamlessly link into and out of installed web apps and the rest of the web as they please.

Service Workers – Going Offline

We said above that another characteristic users often associate with “apps” is their ability to work offline, in the absence of a connection to the Internet. This is historically something the web has done pretty badly at. AppCache was a proposed standard intended for this purpose, but there are many common problems and limitations of that technology which make it difficult or impractical to use in many cases.

A new, much more versatile, proposed standard is called Service Workers. Service Workers allow a script to be registered as managing a slice of the web, even whilst offline, by intercepting HTTP requests to URLs within a specified scope. A Service Worker can keep an offline cache of web resources and decide when to use the offline version and when to fetch a new version over the Internet.

The programmable nature of Service Workers make them an extremely versatile tool in adding app-like capabilities to web content and getting rid of the notion that using the web requires a persistent connection to the Internet. Service Workers have lots of support from multiple browser vendors and you can expect to see them coming to life soon.

The proposed “service_worker” property of the manifest allows a content author to define a Service Worker which should be registered with a specified URL scope when a web app is installed or bookmarked on a device. That means that in the process of installing a web app, an offline cache of web resources can be populated and other installation steps can take place.

Below is our example web app manifest with a Service Worker defined.

http://example.com/myapp/manifest.json:

{
  "name": "My App",
  "icons": [{
    "src": "/myapp/icon.png",
    "sizes": "64x64",
    "type": "image/png"
  }],
  "start_url": "/myapp/index.html"
  "scope": "/myapp"
  "service_worker": {
    "src": "app.js",
    "scope": "/myapp"
  }
}

Packages – The Good, the Bad and the Ugly

There’s a whole category of apps which many people refer to as “web apps” but which are delivered as a package of resources to be downloaded and installed locally on a device, separate from the web. Although these resources may use web technologies like HTML, CSS and Javascript, if those resources are not associated with real URLs on the web, then in my view they are by definition not part of a web app.

The reason this approach is commonly taken is that it allows operating system developers and content authors to side-step some of the current shortcomings of the web platform. Packaging all of the resources of an app into a single file which can be downloaded and installed on a device is the simplest way to solve the offline problem. It also has the convenience that the contents of that package can easily be reviewed and cryptographically signed by a trusted party in order to safely give the app privileged access to system functions which would currently be unsafe to expose to the web.

Unfortunately the packaged app approach misses out on many of the biggest benefits of the web, like its universal and inter-linked nature. You can’t hyperlink into a packaged app, and providing an updated version of the app requires a completely different mechanism to that of web content.

We have seen above how Service Workers hold some promise in finally solving the offline problem, but packages as a concept may still have some value on the web. The proposed “Packaging on the Web” specification is exploring ways to take advantage of some of the benefits of packages, whilst retaining all the benefits of URLs and the web.

This specification does not explore a new security model for exposing more privileged APIs to the web however, which in my view is the single biggest unsolved problem we now have left on the web as a platform.

Conclusions

In conclusion, a look at some of the latest emerging web standards tells us that the answer to the question “what is a web app?” is that a web app is simply a slice of the web which can be used separately from the browser.

With that in mind, web authors should design their content to work just as well inside and outside the browser and just as well offline as online.

Packaged apps are not web apps and are always a platform-specific solution. They should only be considered as a last resort for apps which need access to privileged functionality that can’t yet be safely exposed to the web. New web technologies will help negate the need for packages for offline functionality, but packages as a concept may still have a role on the web. A security model suitable for exposing more privileged functionality to the web is one of the last remaining unsolved challenges for the web as a platform.

The web is the biggest ecosystem of content that exists, far bigger than any proprietary walled garden of curated content. Lots of cool stuff is possible using web technologies to build experiences which users would consider “app like”, but creating a great user experience on the web doesn’t require replicating all of the other trappings of proprietary apps. The web has lots of unique benefits over other app platforms and is unrivalled in its scale, ubiquity, openness and diversity.

It’s important that as we invent cool new web technologies we remember to agree on standards for them which work cross-platform, so that we don’t miss out on these unique benefits.

The web as a platform is here to stay!

2 Comments

Building the Firefox Browser for Firefox OS

2014-08-07 14:41 (UTC)

Re-posted from Mozilla Hacks.

Boot to Gecko

As soon as the Boot to Gecko (B2G) project was announced in July 2011 I knew it something I wanted to contribute to. I’d already been working on the idea of a browser based OS for a while but it seemed Mozilla had the people, the technology and the influence to build something truly disruptive.

At the time Mozilla weren’t actively recruiting people to work on B2G, the team still only consisted of the four co-founders and the project was little more than an empty GitHub repository. But I got in touch the day after the announcement and after conversations with Chris, Andreas and Mike over Skype and a brief visit to Silicon Valley, I somehow managed to convince them to take me on (initially as a contractor) so I could work on the project full time.

A Web Browser Built from Web Technologies

On my first day Chris Jones told me “The next, highest-priority project is a very basic web browser, just a URL bar and back button basically.”

chris_jones

Chris and his bitesize browser, Taipei, December 2011

The team was creating a prototype smartphone user interface codenamed “Gaia”, built entirely with web technologies. Partly to prove it could be done, but partly to find the holes in the web platform that made it difficult and fill those holes with new Web APIs. I was asked to work on the first prototypes of a browser app, a camera app and a gallery app to help find some of those holes.

You might wonder why a browser-based OS needs a browser app at all, but the thinking for this prototype was that if other smartphone platforms had a browser app, then B2G would need one too.

The user interface of the desktop version of Firefox is written in highly privileged “chrome” code using the XUL markup language. On B2G it would need to be written in “content” using nothing but HTML, CSS and JavaScript, just like all the other apps. That would present some interesting challenges.

In the beginning, there was an <iframe>

It all started with a humble iframe, a text input for the URL bar and a go button, in fact you can see the first commit here. When you clicked the go button, it set the src attribute of the iframe to the contents of the text input, which caused the iframe to load the web page at that URL.

first_commit

First commit, November 2011

The first problem with trying to build a web browser using an iframe is that the same-origin policy in JavaScript prevents you accessing just about any information about what’s going on inside it if the content comes from a different origin than the browser itself. In particular, it’s not possible to access the contentWindow property and all of the information that gives access to. This policy exists for good reasons so in order to build a fully functional web browser we would have to figure out a way for a privileged web app to safely poke holes in that cross-origin boundary to get just enough information to do its job, but without creating serious security vulnerabilities or compromising the user’s privacy.

Another problem we came across quite quickly was that many web authors will go to great lengths to prevent their web site being loaded inside an iframe in order to prevent phishing attacks. A web server can send an X-Frame-Options HTTP response header instructing a user agent to simply not render the content, and there are also a variety of techniques for “framebusting” where a web site will actively try to break out of an iframe and load itself in the parent frame instead.

It was quickly obvious that we weren’t going to get very far building a web browser using web technologies without evolving the web technologies themselves.

The Browser API

I met Justin Lebar at the first B2G work week in Taipei in December 2011. He was tasked with modifying Gecko to make the browser app on Boot to Gecko possible. To me Gecko was (and largely still is) a giant black box of magic spells which take the code I write and turn it into dancing images on the screen. I needed a wizard who had a grasp on some of these spells, including a particularly strong spell called Docshell which only the most practised of wizards dare peer into.

justin

Justin at the first B2G Work Week in Taipei, December 2011

When I told Justin what I needed he made the kinds of sounds a mechanic makes when you take your car in for what you think is a simple problem but turns out costing the price of a new car. Justin had a better idea than I did as to what was needed, but I don’t think either of us realised the full scale of the task at hand.

With the adding of a simple boolean “mozbrowser” attribute to the HTML iframe element in Gecko, the Browser API was born. I tried adding features to the browser app and every time I found something that wasn’t possible with current web technologies, I went back to Justin to get him to cast a new magic spell.

There were easier approaches we could have taken to build the browser app. We could have added a mechanism to allow the browser to inject scripts into the iframe and communicate freely with the content inside, but we wanted to provide a safe API which anyone could use to build their own browser app and this approach would be too risky. So instead we built an explicit privileged API into the DOM to create a new class of iframe which could one day become a new standard HTML tag.

Keeping the Web Contained

The first thing we did was to try to trick web pages loaded inside an iframe into thinking they were not in fact inside an iframe. At first we had a crude solution which just ignored X-Frame-Options headers for iframes in whitelisted domains that had the mozbrowser attribute. That’s when we discovered that some web sites are quite clever at busting out of iframes. In the end we had to take other measures like making sure window.top pointed at the iframe rather than its parent so a web site couldn’t detect that it had a parent, and eventually also run every browser tab in its own system process to completely isolate them from each other.

Once we had the animal that is the web contained, we needed to poke a few air holes to let it breathe. There’s some information we need to let out of the iframe in the form of events: when the location, title or icon of a web page changes (locationchange, titlechange and iconchange); when a page starts and finishes loading (loadstart, loadend) and when the security characteristics of the currently loaded page changes (securitychange). This all allows us to keep the address bar and title bar up to date and show a progress indicator.

The browser app needs to be able to navigate the iframe by telling it to goBack(), goForward(), stop() and reload(). We also need to be able to explicitly ask for information like characteristics of the session history (getCanGoBack(), getCanGoForward()) to determine which navigation buttons to display.

With these basics in place it was possible to build a simple functional browser app.

The Prototype

The Gaia project’s first UX designer was Josh Carpenter. At an intensive work week in Paris the week before Mobile World Congress in February 2012, Josh created UI mockups for all the basic features of a smartphone, including a simple browser, and we built a prototype to those designs.

josh

Josh and me plotting over a beer in Paris.

 

The prototype browser app could navigate web content, keep it contained and display basic information about the content being viewed. This would be the version demonstrated at MWC in Barcelona that year.

mwc_browser

Simple browser demo for Mobile World Congress, February 2012

Building a Team

At a work week in Qualcomm’s offices in San Diego in May 2012 I was able to give a demo of a slightly more advanced basic browser web app running inside Firefox on the desktop. But it was still very basic. We needed a team to start building something good enough that we could ship it on real devices.

browser_inception

“Browser Inception”, San Diego May 2012

San Diego was also where I first met Dale Harvey, a brave Scotsman who came on board to help with Gaia. His first port of call was to help out with the browser app.

dale

Dale Getting on Board in San Diego, May 2012

One of the first things Dale worked on was creating multiple tabs in the browser and even adding a screenshotting spell to the Browser API to show thumbnails of browser tabs (I told you he was brave).

By this time we had also started to borrow Larissa Co, a brilliant designer from the Firefox team, to work on the interaction design and Patryk Adamczyk, formerly of RIM, to work on the visual design for the browser on B2G. That was when it started to look more like a Firefox browser.

start_page

Early UI Mockup, July 2012

Things that Pop Up

Web pages like to make things pop up. For a start they like to alert(), prompt() or confirm() things with you. Sometimes they like to open() a new browser window (and close() them again), open a link in a _blank window, ask you for a password, ask for your permission to do something, ask you to select an option from a menu, open a context menu or confirm re-sending the contents of a form.

alert

An alert(), version 1.0

All of this required new events in the Browser API, which meant more spells for Justin to cast.

Scroll, Pan and Zoom

Moving around web pages on web devices works a little differently from on the desktop. Rather than scroll bars or a scroll wheel on a mouse it uses touch input and a system called Asynchronous Pan and Zoom to allow the user to pan around a web page by dragging it and scrolling it using “kinetic scrolling” which feels like it has some physics to it.

The first implementation of kinetic scrolling was written in JavaScript by Frenchman and Gaia leader Vivien Nicolas, specifically for Gaia, but it would later be written in a cross-platform way in Gecko to unify the code used on B2G and Android.

One of the trickier interactions to get right was that we wanted the address bar to hide as you scrolled down the page in order to make more room for content, then show again when you scroll back to the top of the page.

This required adding asyncscroll events which tapped directly into the Asynchronous Pan and Zoom code so that the browser knew not only when the user directly manipulated the page, but how much it scrolled based on physics, asynchronously from the user’s interaction.

Storing Stuff

One of the most loved features of Firefox is the “Awesomebar”, a combined address bar, search bar (and on mobile, title bar) which lets you quickly get to the content you’re looking for. You type a few characters and immediately start to see matching web pages from your browsing history, ranked by a “frecency” algorithm.

On the desktop and on Android all of this data is stored in the “Places” database as part of privileged “chrome” code. In order to implement this feature in B2G we would need to use the local storage capabilities of the web, and for that we chose IndexedDB. We built a Places database in IndexedDB which would store all of the “places” a user visits on the web including their URL, title and icon, and store all the times the user visited that page. It would also be used to store the users bookmarks and rank top sites by “frecency”.

awesomebar

Awesomebar, version 1.0

Clearing Stuff

As you browse around the web Gecko also stores a bunch of data about the places you’ve been. That can be cookies, offline pages, localStorage, IndexedDB databases and all sorts of other bits of data. Firefox browsers provide a way for you to clear all of this data, so methods needed to be added to the Browser API to allow this data to be cleared from the browser settings in B2G.

settings

Browser settings, version 1.0

Handling Crashes

Sometimes web pages crash the browser. In B2G every web app and every browser tab runs in its own system process so that should the worst happen, it will only cause that one window/tab to crash. In fact, due to the memory constraints of the low-end smartphones B2G would initially target, sometimes the system will intentionally kill a background app or browser tab in order to conserve memory. The browser app needs to be informed when this happens and needs to be able to recover seamlessly so that in most cases the user doesn’t even realise a process was killed. Events were added to the Browser API for this purpose.

crash

Crashed tab, version 1.0

Talking to Other Apps

Common use cases of a mobile browser are for the user to want to share a URL using another app like a social networking tool, or for another app to want to view a URL using the browser.

B2G implemented Web Activities for this purpose, to add a capability to the web for apps to interact with each other, but in an app-agnostic way. So for example the user can click on a share button in the browser app and B2G will fire a “share URL” Web Activity which can then be handled by any installed app which has registered to handle that type of Web Activity.

share

Share Web Activity, version 1.2

Working Offline

Despite the fact that B2G and Gaia are built on the web, it is a requirement that all of the built-in Gaia apps should be able to function offline, when an Internet connection is unavailable or patchy, so that the user can still make phone calls, take photos and listen to music etc.. At first we started to use AppCache for this purpose, which was the web’s first attempt at making web apps work offline. Unfortunately we soon ran into many of the common problems and limitations of that technology and found it didn’t fulfill all of our requirements.

In order to ship version 1.0 of B2G on time, we were forced to implement “packaged apps” to fulfill all of the offline and security requirements for built-in Gaia apps. Packaged apps solved our problems but they are not truly web apps because they don’t have a real URL on the Internet, and attempts to standardise them didn’t get much traction. Packaged apps were intended very much as a temporary solution and we are working hard at adding new capabilities like ServiceWorkers, standardised hosted packages and manifests to the web so that eventually proprietary packaged apps won’t be necessary for a full offline experience.

offline

Offline, version 1.4

Spit and Polish

Finally we applied a good deal of spit and polish to the browser app UI to make it clean and fluid to use, making full use of hardware-accelerated CSS animations, and a sprinkling of Firefoxy interaction and visual design to make the youngest member of the Firefox browser family feel consistent with its brothers and sisters on other platforms.

Shipping 1.0

At an epic work week in Berlin in January 2013 hosted by Deutsche Telekom the whole B2G team, including engineers from multiple competing mobile networks and device manufacturers, got together with the common cause of shipping B2G 1.0, in time to demo at Mobile World Congress in Barcelona in February. The team sprinted towards this goal by fixing an incredible 200 bugs in one week.

berlin

Version 1.0 Team, Berlin Work Week, January 2013

In the last few minutes of the week Andreas Gal excitedly declared “Zarro Gaia Boogs”, signifying version 1.0 of Gaia was complete, with the rest of B2G to shortly follow over the weekend. Within around 18 months a dedicated team spanning multiple organisations had come together working entirely in the open to turn an empty GitHub repository into a fully functioning mobile operating system which would later ship on real devices as Firefox OS 1.0.1.

zarro_boogs

Zarro Gaia Boogs, January 2013

v1

Browser app v1.0

So having attended Mobile World Congress 2012 with a prototype and a promise to deliver commercial devices into the market, we were able to return in 2013 having delivered on that promise by fully launching the “Firefox OS” brand with multiple devices on multiple mobile networks with a launch that really stole the show at the biggest mobile conference in the world. Firefox OS had arrived.

mwc

Mobile World Congress, Barcelona, February 2013

1.x

Firefox OS 1.1 quickly followed and by the time we started working on version 1.2 the project had grown significantly. We re-organised into autonomous agile teams focused on product areas, the browser app being one. That meant we now had a dedicated team with designers, engineers, a test engineer, a product manager and a project manager.

browser_team

The browser team, London work week, July 2013

Firefox OS moved to a rapid release “train model” of development like Firefox, where a new version is delivered every 12 weeks. We quickly added new features and worked on improving performance to get the best out of the low end hardware we were shipping on in emerging markets.

v1.4

Browser app v1.4

“Haida”

Version 1.0 of Firefox OS was very much about proving that we could build what already exists on other smartphones, but entirely using open web technologies. That included a browser app.

Once we’d proved that was possible and put real devices on shelves in the market it was time to figure out what would differentiate Firefox OS as a product going forward. We wanted to build something that doesn’t just imitate what’s already been done, but which plays to the unique strengths of the web to build something that’s true to Mozilla’s DNA, is the best way to experience the web, and is the platform that HTML5 deserves.

Below is a mockup I created right back towards the start of the project at the end of 2011, before we even had a UX team. I mentioned earlier that the Awesomebar is a core part of the Firefox experience in Firefox browsers. My proposal back then was to build a system-wide Awesomebar which could search the whole device, including your apps and their contents, and be accessible from anywhere in the OS.

b2g_awseomescreen

Very early mockup of a system-wide Awesomebar, December 2011

At the time, this was considered a little too radical for version 1.0 and our focus really needed to be on innovating in the web technology needed to build a mobile OS, not necessarily the UX. We would instead take a more conservative approach to the user interface design and build a browser app a lot like the one we’d built for Android.

In practice that meant that we in fact built two browsers in Firefox OS. One was the browser app which managed the world of “web sites” and the other was the window manager in the system app which managed the world of “web apps” .

In reality on the web there isn’t so much of a distinction between web apps and web sites – each exists on a long continuum of user experience with a very blurry boundary in the middle.

In March 2013, with Firefox OS 1.0 out of the door, Josh Carpenter put me in touch with Gordon Brander, a member of the UX team who had been thinking along the same lines as me. In fact Gordon being as much of an engineer as he is a designer, had gone as far as to write a basic prototype in JavaScript.

rocketbar_prototype

Gordon’s Rocketbar Prototype, March 2013

Gordon and I started to meet weekly to discuss the concept he had by then codenamed “Rocketbar”, but it was a bit of a side project with a few interested people.

In April 2013 the UX team had a summit in London where they got together to discuss future directions for the user experience of Firefox OS. I was lucky enough to be invited along to not only observe but participate in this process, Josh being keen to maintain a close collaboration between Design and Engineering.

We brainstormed around what was unique about the experience of the web and how we might create a unique user experience which played to those strengths. A big focus was on “flow”, the way that we can meander through the web by following hyperlinks. The web isn’t a world of monolithic apps with clear boundaries between them, it is an experience of surfing from one web site to another, flowing through content.

ux_workshop

Brainstorming session, London, April 2013

In the coming weeks the UX team would create some early designs for a concept (eventually codenamed “Haida”) which would blur the lines between web apps and web sites and create a unique user experience which flows like the web does. This would eventually include not only the “Rocketbar”, which would be accessible across the whole OS and seamlessly adapt to different types of web content, but also “sheets”, which would split single page web apps into multiple pages which you could swipe through with intuitive edge gestures. It would also eventually include a content model based around live apps which you can surf to, use, and then bookmark if you choose to, rather than monolithic apps which you have to install from a central app store before you can use them.

In June 2013 a small group of designers and engineers met in Paris to develop a throwaway prototype of Haida, to rapidly iterate on some of the more radical concepts and put them through user testing.

haida_team

Haida Prototyping, Paris, June 2013

josh_and_gordon

Josh and Gordon working in a highly co-ordinated fashion, Paris, June 2013

haida_engineers

Wizards at work, Paris, June 2013

2.x and the Future

Fast forward to the present and the browser team has been merged into the “Systems Front End” team. The results of the Haida prototyping and user testing are slowly starting to make their way into the main Firefox OS product. It won’t happen all at once, but it will happen in small pieces as we iterate and learn.

In version 2.0 of Firefox OS the homescreen search feature from 1.x will be replaced with a new search experience developed in conjunction with a new homescreen, implemented by Kevin Grandon, which will lay the foundations for “Rocketbar”. In version 2.1 our intention is to completely merge the browser app into the system app so that browser tabs become “sheets” alongside apps in the task manager and the “Rocketbar” is accessible from anywhere in the OS. The Rocketbar will adapt to different types of web content and shrink down into the status bar when not in use. Edge gestures will allow you to swipe between web apps and browser windows and eventually apps will be able to spawn multiple sheets.

rocketbar

UI Mockups of Rocketbar in expanded and collapsed state, July 2014

In parallel we see the evolution of web standards around manifests, packages and webviews and ongoing discussions around what defines the scope of an “app”.

Version 1.x of Firefox OS was built with web technologies but still has quite a similar user experience to other mobile platforms when it comes to installing and using apps, and browsing the web. Going forward I think you can expect to see the DNA of the web come through into the user interface with a unified experience which breaks down the barriers between web apps and web sites, allowing you to freely flow between the two.

Firefox OS is an open source project developed completely in the open. If you’re interested in contributing to Gaia, take a look at the “Developing Gaia” page on MDN. If you’re interested in creating your own HTML5 app to run on Firefox OS take a look at the “App Center“.

No Comments

Introducing DRD Pi

2013-04-07 00:14 (UTC)

I’m trying to build a Raspberry Pi powered robot based on the DRDs from Farscape, I thought I’d blog my progress.

DRD

DRDs or “Diagnostic Repair Drones” are robots from the cult science fiction series Farscape. They carry out various functions aboard a leviathian (a species of living biomechanoid spaceship) including repairing and maintaining the ship. They’re ovoid in shape and they have two moving eye stalks and all sorts of tools like a robotic claw and a plasma welder.

DRD

Here’s some video footage from the series to give you an idea of what these little guys get up to:

DRD Kit

The original DRDs were designed and built by the extremely talented folks at the Jim Henson Creature Shop in London (yes Jim Henson as in the Muppets!). They built lots of different variations of the robot over the years to be used in shooting different scenes for the show, but to my knowledge they’ve never released any designs.

I assumed I was going to have to painstakingly design a 3D computer model of one based on frame grabs from my DVDs of the series. I then planned to track down someone with a CNC router and a vacuum forming machine and persuade them to let me use them. Either that or find someone with an industrial sized 3D printer!

Luckily I came across a special effects company in the US who sells a kit to build a model of a DRD. The model is made from hollow cast fiberglass and resin and comes with ribbed plastic for the eye stalks, eye pieces with clear lenses, two parts of a claw and some colourful wires to make it look the part.

drd_kit

The kit isn’t perfect. The size, shape and proportions aren’t quite right and the finish is a bit rough but it’s good enough for my purposes. The part I’m really interested in is the robotics so I’m grateful that someone has already done the work for me on the basic shell.

The web site provides video tutorials on how to build the model and then how to put LEDs in the eyes and mount an remote controlled car underneath to make it move about in a bit of a crude fashion.

We can be a bit more sophisticated than that.

Raspberry Pi

The Raspberry Pi is a credit-card sized computer developed in the UK by the not-for-profit Raspberry Pi Foundation to promote the teaching of programming in schools. It’s a single-board computer with a 700Mhz ARM processor and 512MB RAM, boots off an SD card and costs only around £30.

This is my Raspberry Pi:

raspberry_pi

Gertboard

The Gertboard is an expansion board which attaches to the Raspberry Pi via its GPIO pins and helps when experimenting with interfacing the Pi with the outside world. It comes with an Arduino compatible AVR microcontroller, analogue to digital converters, digital to analogue converters, a motor controller, push buttons, LEDs and much more.

gertboard

Booting the Pi

The Raspberry Pi can boot Linux from an SD card and the most popular distribution is Raspbian which is a Debian-derivative. You can download an image and flash it to an SD card, or even buy an SD card with it already loaded.

To boot the Raspberry Pi all you need to do is insert your Raspbian SD card, plug it into a TV via either the HDMI port or the composite video port and power it up by plugging it into a Micro USB phone charger.

Here’s my Raspberry Pi booted and plugged into an old CRT TV:

raspberry_pi_and_tv

 

Logging In Remotely

It’s cool that I can plug the Raspberry Pi into a TV, but I don’t want to be squinting at an old portable TV or sitting in the lounge next to my big flatscreen TV all the time I’m programming the robot, so I want to be able to log in remotely. Also, my plan is to build a web interface to control the robot over WiFi, so it’s going to need to connect to a network at some point.

First I plugged a USB keyboard into the Raspberry Pi and an ethernet cable to connect it to my network. The SSH daemon is already started by default, but I wanted to set a static IP address so that I always knew what to log in to.

I logged into the Raspberry Pi locally (the default username is pi and the password is raspberry) and edited the network configuration using the vi text editor.

$ sudo vi /etc/network/interfaces

I provided the following configuration to assign a static IP address of 192.168.1.42 on my local network:

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
address 192.168.1.42
netmask 255.255.255.0
gateway 192.168.1.1

Then restart the network interface with:
$ sudo ifdown -a
$ sudo ifup -a

Then check that I’m connected to the network, and the Internet by pinging Google.

$ ping google.com

I see that I’m successfully connected, so I can now log into the Raspberry Pi remotely using its new static IP.

From my desktop Linux box I type:

$ ssh pi@192.168.1.42

type in the password “raspberry”, and voilà! I’m logged in.

ssh

What’s Next

I hope you weren’t expecting to see a finished robot! There’s a very long way to go yet.

If you desperately wanted to see a finished robot, here’s a picture of the last one I worked on, a line following robot we built at university powered by a PIC microcontroller.

BEAST

Next I want to start playing around with the Gertboard and and make LEDs blink on and off from Python.

No Comments

Introducing Krellian, Webian's New Sponsor

2011-10-28 23:34 (UTC)

Reposted from webian.org

Firstly, thanks for the incredible continued contributions from the Webian community and for all the work you've done on Webian Shell, which has now had more than 95,000 downloads!

Introducing Krellian

This week I left my job as Product Manager of Clinked at Rabbitsoft to start a software consultancy called Krellian.

Through Krellian I will be able to continue to lead the Webian project, and I will also be taking up a new contract with the Mozilla Corporation to work with them on Boot to Gecko (B2G).

Like me and the other members of the Webian community, Mozilla believes that the open web can displace proprietary, single-vendor stacks for application development. The B2G project will include prototype APIs for exposing device and OS capabilities to web content, a privilege model to safely expose these new capabilities, a complete "low-level substrate" for Android-compatible devices and a collection of web apps to prioritise and prove the power of the platform.

Benefits to Webian

The potential benefits for Webian are enormous. Webian Shell was already hitting limitations of what is currently possible with Mozilla Chromeless and this new work on the core Mozilla platform promises to make many more of Webian's goals possible. While B2G initially focuses on the mobile space, Webian can focus on nettop and netbook form factors and perhaps eventually the two projects could even converge.

Sponsorship from Krellian will provide the ongoing resources necessary for running the Webian project and ensure that it remains free and open source.

I'm excited about this new chapter in Webian's story and believe more strongly than ever in the future of the open web.

No Comments

Keeping Web Apps Open

2011-10-10 16:09 (UTC)

Reposted from webian.org

The Universal App Platform

The best thing about web apps is that they can run on any platform. This is because they use a set of open standards which are universally supported. The latest incarnations of these standards like HTML5 and CSS3 are more powerful than ever and it seems inevitable to me that the web will eventually win out as the open, universal app platform.

The current generation of popular desktop operating systems like Windows 7, OS X and Linux and mobile operating systems like iOS, Android and Windows Phone treat web apps as second class citizens, inferior to their native platform-specific counterparts. But that's all set to change.

Microsoft Windows 8, Google Chrome OS, The Linux Foundation's Tizen and Mozilla B2G will all join Webian in putting web apps centre stage, as the preferred category of app on their respective platforms.

Fragmentation

Thanks to HTML5 and related standards web apps are already pretty powerful, but there is still some way to go before web apps can do everything that native apps can do. On mobile platforms new APIs are needed for access to telephony, messaging and cameras for example, and some kind of "app store" style distribution and installation mechanism is likely to be needed to provide a more app-like experience. "Installing" a web app is a little like bookmarking a web site and can provide an opportunity for the app to ask the user for all the permissions it needs up-front, download any resources which will be needed for offline use and perhaps add an shortcut icon to an app list.

Because there aren't currently established standards for all of these features, each platform is intially rolling its own approach. Apps for Chrome OS can only be found via Google's own "Chrome Web Store", you have to have a Google Account to install them and many of the apps found there will only work in Chrome. Windows 8 will have a set of APIs for its new Metro user interface, but apps written for Metro won't necessarily work on other platforms. We don't yet know much about Tizen, but previous mobile platforms like HP's WebOS and the MeeGo Web Runtime have also had platform-specific APIs.

An Open Approach

Mozilla is also working on a new set of web APIs, but in line with their mission of promoting openness on the web they're doing so in a more open way. You can see the progress of their APIs for telephonymessaging (SMS), batteryvisibilitysettingspower management and contacts on Bugzilla. These new APIs on Mozilla's core platform are the first step towards a new mobile OS from Mozilla called B2G or "Boot to Gecko". B2G will have a highly customisable UI built entirely with web technologies and will allow for the installation of Open Web Apps, as proposed by Mozilla Labs. With Open Web Apps, anyone can run their own app store or directory and the aim is to create cross-platform installable web apps.

Mozilla is also collaborating with Google on "Web Activities" or "Web Intents" which could provide a standard mechanism for apps to call other apps for certain functions. This collaboration is a promising sign because Mozilla can not define standards on their own and will need to co-operate with other browser & OS vendors.

Towards Standards

There are already standardisation efforts in this area at the W3C such as the Device APIs Working Group and the Widget Packaging and XML Configuration recommendation, but W3C recommendations are not always in line with what the implementors (browser & OS vendors) are doing.

For example, W3C Widgets provide a mechanism for installing a "widget" locally on a device by packaging up all of its resources in a zip file, including an XML manifest file. The specification was used for widgets in the Opera browser but when the MeeGo Web Runtime tried to use it as a method of installing fully fledged web apps it didn't work out very well.

Mozilla and Google are taking a different approach to installable web apps, both choosing to use a JSON manifest file and rely to a large extent on HTML5's offline capabilities for caching and updating offline resources. Even these two similar implementations have subtle differences which make them incompatible with each other so there's a real need for standardisation in this area if we're to have cross-platform installable web apps.

Hosted vs. Packaged

One slightly odd trend in the standardisation of "web apps" is the tendency to focus on "packaging" apps rather than "hosting" them. This seems to be an attempt to emulate the way mobile and desktop apps currently work by bundling up all of the app's resource in a "package" which is downloaded and installed locally. The whole package then needs to be updated every time any of the resources change. This seems odd to me because this isn't the way the web usually works and it doesn't seem very web-like at all.

The fact that an individual resource can be identified by a URI is what makes it a part of the web, so to package resources up in one big bundle to be downloaded and used locally seems to break the web paradigm and therefore these apps are not "web apps" in my view, just apps downloaded over the web. This is how W3C widgets work and it's also how Chrome "packaged" apps work and seems to be the direction of the Wholesale Apps Community, the EU funded Webinos project and the Native Web Apps community on the W3C web site.

For an app to be part of the web, I think all of its resources (HTML, CSS, JavaScript, images etc.) must be identifiable by a URI. Resources which are needed when operating offline can be marked as such in an HTML5 cache manifest, which then also provides an automatic update mechanism when these resources are changed. This maintains one of the great characteristics of web apps which is their ability to be updated seamlessly with new features and bugfixes without the user having to manually upgrade to a new version. Mozilla Open Web Apps seem to work in this way, and so do Chrome's "hosted" apps.

Conclusions

It seems that web apps will be the preferred category of apps for a whole new generation of operating systems, but in order for these web apps to remain cross-platform new open standards must be agreed upon for APIs which give web apps all the power of native apps.

A mechanism for the discovery, installation and updating of hosted installable web apps and a trust framework which allows for secure open distribution is also something which I think desperately needs standardisation if the vision of a universal open web app platform is to be realised.

No Comments

Announcing Webian Shell 0.1

2011-05-27 17:15 (UTC)

(Reposted from the Webian blog) Webian Shell is a full screen web browser for devices that don't need a desktop. As of today you can download a prototype which you can try for yourself.

A Graphical Shell for the Web

If you’re anything like me then you’ll find that most of the stuff you do on your PC these days happens in a web browser and the desktop environment you used to depend on is now just getting in your way. Modern web applications can do just about everything traditional desktop applications can do (like flashy graphics, sound, video and offline storage) so often all you really need is the browser. The idea of the Webian Shell project is to replace your computer’s interface with something much simpler, which treats web applications as first class citizens and does away with all the un-necessary clutter.

Webian Shell screenshot
See a screencast on YouTube

Rapid Prototyping with Mozilla Chromeless

Shell started as a simple design concept with a few static mockups, but when Mozilla Chromeless came along it was suddenly possible to rapidly develop a working prototype using standard web technologies like HTML, CSS and JavaScript. As a member of the wider Mozilla community this was an opportunity I couldn’t resist so I quickly got to work on putting together a prototype using the technologies I already knew from web development. This initial early release really just gives you a minimalist, full screen, tabbed web browser with a clock and a rather empty looking home screen. I hope this is enough to convey the basic idea behind the project and that with the open source community's help we can quickly iterate this prototype to encompass lots of exciting ideas about what living on the web could really be like.

Future Directions

These are some of my ideas for future directions to take the prototype, but I’d love to hear your ideas.

Download the Prototype

You can find out more on the Webian web site or go straight ahead and download and install Webian Shell 0.1 on your Mac, Windows or Linux computer today. I’d really love to hear your feedback! You can propose ideas, report bugs and ask questions on GetSatisfaction, or join the Google Group to take part in the conversation. You can also follow Webian on Twitter or Identi.ca, like it on Facebook and check out the channel on YouTube.

No Comments

My Chromercial

2010-12-15 14:22 (UTC)

Some Google Chrome engineers uploaded a video they called a "Chromercial" to promote Chrome OS, much to the marketing department's dismay! They asked for video responses where you give your own take on Chrome OS.

You can watch my video response on YouTube here.

View the video on YouTube 

I originally created the artwork I used in this video for my "Graphical Shell for the Web" design concept which basically portrays the same idea as Chrome OS (but was published before Chrome OS was released!)

I created the artwork using Inkscape and put the video together using PiTiVi. I used Creative Commons licensed music and sound effects and licensed the video itself under a Creative Commons Non-commercial Sharealike license so you can remix & share it as much as you like. You can download the source png slides and sound files (along with the completed video in ogg format) here (I also have the SVG source of the images if anyone wants it).

 

No Comments

Installable Web Apps

2010-10-20 11:47 (UTC)

When I first heard about Google's and then Mozilla's concepts of "installable web apps" I was a little dubious. Surely the whole benefit of web applications is that they don't need to be installed!

But if you think of installable web apps as a convergence of the "app store" and "cloud computing" models (arguably the two most significant trends in the software industry today) it actually makes quite a lot of sense.

The diversity of different software platforms available for smartphones, tablets, smart TVs and PCs is great for encouraging competition and driving innovation – but it also causes problems for both app developers and consumers. An app written for Apple's iOS for example will not run on Android, Blackberry OS, WebOS, Meego, Bada, Symbian or any other competing platform. For consumers this is confusing and frustrating because unless they allow themselves to be locked in to a single vendor they may have to re-purchase different apps for different devices. For app developers it's even more frustrating because they're faced with a bewildering array of platforms to target their software at.

Web applications have different problems with limited discoverability and access to hardware. To a web browser web applications are no different to any other web site which means that the apps can easily get lost in the ocean of content on the web. Web apps also traditionally have very limited default access to client-side hardware for security reasons which limits their capabilities when compared to native applications.

Installable web apps can combine the best parts of web applications (the benefits of cloud computing and accessiblity from any device) with the best parts of the app store model (easy distribution and discovery of software which has full access to the device's hardware). This new breed of apps could leverage web technologies to reach the whole range of competing platforms with a single application which can be distributed via a range of competing app stores. By "installing" a web application the user can pre-approve access to local hardware like offline storage, accelerated graphics and geolocation to allow the app to make the most of the hardware it's running on and the user can keep a local repository of all the applications they use.

Both developers and consumers only have to worry about one kind of application running on one open platform (the web), but still have a choice of hardware and app stores. In this model the only people who lose out are the corporations who seek to lock consumers into their own products and services.

It could take a long time for installable web apps to become the predominant model of software distribution, but the idea of converging cloud computing with the app store model makes the idea very appealing and might go some way to explaining why Google currently has both the Android and Chrome OS operating systems in development. Each OS is approaching the convergence from a different direction whilst maximising on the market potential of what can already be done today.

I hope that standards for "installable web apps" can be agreed upon sooner rather than later so that if this model does become a growing trend, the new breed of apps will not suffer from the same fragmentation as other platforms do today.

No Comments

Rise of the Smart TV

2010-09-20 13:35 (UTC)

Reposted from webian.org 

Apart from netbooks, tablets and smartphones another key target device for accessing Webian Home Server will be TVs. With big players having recently announced their entries into the smart TV space, this blog post explores their different approaches and the unique challenges of designing for the TV.

Google TV

Many companies have tried to combine web and TV experiences before, but it's never really caught on. Google believes that their approach will be a success because they aren't trying to re-create the web for the TV or make you choose between the web or TV, instead they want to give you access to the whole of the existing web and all the channels you already receive, as well as allowing you to install apps from the Android marketplace all from one device.

Google TV

Google TV will allow all of this to be done via a 10ft interface with a remote control including a full QWERTY keyboard and pointing device. The software stack will include the Android operating system, Chrome browser, Flash and a new IP-based protocol to interface with DVR boxes allowing you search TV listings and schedule recordings.

Rather than a releasing a single Google branded device, Google has defined a standard hardware and software platform which will be implemented by multiple device manufacturers. Among the first will be an HDTV and BluRay player from Sony and a set-top box from Logitech which are expected this autumn.

Apple TV

Apple has recently announced the release of the latest generation of Apple TV which takes a very different approach to that of Google. The original version of Apple TV was released in 2006 but didn't prove to be a huge success. In the keynote announcing the new version, Steve Jobs explained that Apple had learnt a lot from the customers who did buy the original set-top device and that this time they think they've cracked it.

Apple TV

Apple's observations were that consumers want premium HD content delivered to their TV, but they don't want a "computer" on their TV or to have to manage storage on their TV or sync content with other devices. Their response is a new generation of Apple TV which is a smaller streaming-only device which streams content from other Apple products (iPod, iPhone, iPad or iMac) over the home network or from Apple's partners over the Internet with a rental model.

The new Apple TV is characteristically proprietary but well designed, with a slick custom UI for a closed list of tightly integrated online services. Apple has chosen a much simpler remote control with just a few buttons and opts for an interaction style much more familiar to TV users.

Boxee

Another smart TV option comes in the form of the Boxee Box from a small startup in the US. You can already download a beta of the Boxee software to your PC but with "Boxee Box", D-Link has manufactured a piece of hardware dedicated to using Boxee on your TV which is available to pre-order now.

The Boxee software has an interesting heritage (having been derived from the open source XBMC project) and has a great UI which is well suited to a 10ft interface. The remote control which comes with the D-Link device has a simple ordinary looking remote on one side, but you can flip it over to reveal a full QWERTY keyboard if you need it.

Boxee is pretty rich on features, it will let you stream films, TV shows, music and photos in a large variety of formats over your home network and adds a social dimension by allowing you to share what media you're viewing with your friends. Boxee will also have a Webkit-based web browser built in and is extensible through the development of apps.

YouView

Whilst big and small tech companies in the US launch their entires into the TV space, big media companies in the UK have been collaborating to create a new platform called YouView. YouView is billed as a replacement for the popular free-to-view TV "FreeView" boxes currently popular on this side of the pond.

YouView (previously codenamed Project Canvas) will provide the familiar digital TV channels but will also provide on-demand catch-up content from all of the major UK channels on your TV. YouView will also let you pause & rewind live TV, it will provide pay-TV options and will allow developers to create apps for the platform.

The BBC's heavy involvement in YouView is controversial with some other media companies who complain that the license fee funded corporation should not be using their resources to develop this platform which they argue damages competition for smaller players. Although the collaboration of so many large media and communications companies makes YouView a compelling choice in the UK, it doesn't follow existing standards common in Europe and the rest of the world.

Currently YouView is publicly little more than a collection of technical documents available on the web, but the first devices are scheduled to be launched in Q1 2011 and are likely to make a big splash in the UK.

Other Players

The products described above seem to have caught the attention of the media and blogosphere recently, but there are of course many other options out there. Other big players in this space include Microsoft's older Zune and Mediaroomplatforms for the XBox 360 games console and set-top boxes respectively and Samsung's own app platform for their high end TVs. There are also many open source projects like MythTV which provide a geekier way to create your own smart DVR.

Conclusions

The four different products described here take quite different approaches to the smart TV problem but also have similarities. I very much support Google's open approach but they could learn a thing or two from Apple's user experience, I'm really not sure a QWERTY keyboard and pointing device is going to catch on in the living room – the TV is not a PC. Boxee provides a compelling alternative but their app platform may struggle faced with competition from Android and others. YouView will be difficult to ignore in the UK but it provides yet another platform to develop apps for.

One thing all four have in common is the idea that the TV is for streaming content from other sources, not as the media centre for the home. The smart TV solutions presented here allow you to stream media from the Internet or from other devices like PCs, tablets and smartphones on the home network to your TV – but there is no central store for your whole media collection which can be accessed 24/7 by all of these devices.

If there was such a centralised media store then creating an app for every smart TV platform, mobile platform and tablet platform in order to access that media would seem impractical. The common denominator among all of these devices (with the exception of Apple TV which currently offers no opportunities for third party development) is the web, specifically HTML5. A single HTML5 web application hosted in the home could allow access to your entire media collection 24/7 from all of these devices.

Central Home Media Store

The challenge for such web applications will be to create a user interface which works well on such a range of form-factors – from a small multi-touch smartphone through to the 10ft interface of a smart TV with a simple remote control. This will require a new breed of applications which can be controlled in a variety of ways and can adapt to a range of different screen sizes. Some examples of these experimental new types of web applications are YouTube LeanbackGoogle Reader Play and Clicker but it's likely to take a long while for best practices to emerge.

With approximately 1 billion PC users, 2 billion mobile users and 4 billion TV users worldwide – web applications with the potential to support all of these devices would seem very compelling indeed!

This is why the Webian project will strive to create a graphical web interface which provides an enjoyable user experience across all of these form factors.

No Comments

Webian Project Inception

2010-08-24 11:41 (UTC)

A quick update on some foundations being laid for the Webian project.

The project scope has been set. Broadly this covers managing photos, music and videos but with some potential future directions and specific exclusions as well. Some very high level requirements have been developed for the first release with a focus on photo management.

The focus for the first release will be on making a really good job of photo management using a slick HTML5 interface. That includes uploading photos, organising them into albums, tagging them, subscribing to photo feeds, browsing and searching.

A simple initial architecture design describes an HTML5 interface served from a Django-based Python web application. The next step is to build some technical prototypes to explore some key technologies and UI prototypes to validate the key high level use cases. This will mark the end of the "inception phase" of the project and next will be the "elaboration phase" when the architecture design will be tested with a working architecture prototype.

Talking of phases, the project plan approximates the Agile Unified Process but with a particularly user-centered flavour. The AUP is kind of an Agile version of the heavyweight Rational Unified Process used in enterprise software development. The AUP is described as being "serial in the large, iterative in the small" and aims to take a lightweight, iterative approach to software development, but with the recognition of distinct phases of the project. I've chosen to experiment with this methodology to try and get the best out of both user-centered design and agile development. If it doesn't work, I'll switch to something else!

If you want to get involved in the Webian project then sign up to the mailing list and introduce yourself!

No Comments
SubscribeSubscribe (RSS)