(re-posted from a page I created on the Mozilla wiki on 17th December 2014)
The per-OS app store model has resulted in a market where a small number of OS companies have a large amount of control, limiting choice for users and app developers. In order to get things done on mobile devices users are restricted to using apps from a single app store which have to be downloaded and installed on a compatible device in order to be useful.
The idea of pinned apps is to turn the apps model on its head by making apps something you discover simply by searching and browsing the web. Web apps do not have to be installed in order to be useful, “pinning” is an optional step where the user can choose to split an app off from the rest of the web to persist it on their device and use it separately from the browser.
”If you think of the current app store experience as consumers going to a grocery store to buy packaged goods off a shelf, the web is more like a hunter-gatherer exploring a forest and discovering new tools and supplies along their journey.”
A Web App Manifest linked from a web page says “I am part of a web app you can use separately from the browser”. Users can discover web apps simply by searching or browsing the web, and use them instantly without needing to install them first.
”App discovery could be less like shopping, and more like discovering a new piece of inventory while exploring a new level in a computer game.”
If the user finds a web app useful they can choose to split it off from the rest of the web to persist it on their device and use it separately from the browser. Pinned apps can provide a more app-like experience for that part of the web with no browser chrome and get their own icon on the homescreen.
”For the user pinning apps becomes like collecting pin badges for all their favourite apps, rather than cluttering their device with apps from an app store that they tried once but turned out not to be useful.”
Once a pinned app is registered as managing its own part of the web (defined by URL scope), any time the user navigates to a URL within that scope, it will open in the app. This allows deep linking to a particular page inside an app and seamlessly linking from one app to another.
”The browser is like a catch-all app for pages which don’t belong to a particular pinned app.”
Pinning an app could download its contents to the device to make it work offline, by registering a Service Worker for the app’s URL scope.
”Pinned apps take pinned tabs to the next level by actually persisting an app on the device. An app pin is like an anchor point to tether a collection of web pages to a device.”
A web app is a collection of web pages dedicated to a particular task. You should be able to have multiple pages of the app open at the same time. Each app could be represented in the task manager as a collection of sheets, pinned together by the app.
”Exploding apps out into multiple sheets could really differentiate the Firefox OS user experience from all other mobile app platforms which are limited to one window per app.”
Even in a world without app stores there would still be a need for a curated collection of content. The Marketplace could become less of a grocery store, and more of a crowdsourced travel guide for the web.
”If a user discovers an app which isn’t yet included in the guide, they could be given the opportunity to submit it. The guide could be curated by the community with descriptions, ratings and tags.”
What value (the importance, worth or usefulness of something) does your idea deliver?
The pinned apps concept makes web apps instantly useful by making “installation” optional. It frees users from being tied to a single app store and gives them more choice and control. It makes apps searchable and discoverable like the rest of the web and gives developers the freedom of where to host their apps and how to monetise them. It allows Mozilla to grow a catalogue of apps so large and diverse that no walled garden can compete, by leveraging its user base to discover the apps and its community to curate them.
What technological advantage will your idea deliver and why is this important?
Pinned apps would be implemented with emerging web standards like Web App Manifests and Service Workers which add new layers of functionality to the web to make it a compelling platform for mobile apps. Not just for Firefox OS, but for any user agent which implements the standards.
Why would someone invest time or pay money for this idea?
Users would benefit from a unique new web experience whilst also freeing themselves from vendor lock-in. App developers can reduce their development costs by creating one searchable and discoverable web app for multiple platforms. For Mozilla, pinned apps could leverage the unique properties of the web to differentiate Firefox OS in a way that is difficult for incumbents to follow.
Web App Manifest
A manifest is linked from a web page with a link relation:
<link rel=”manifest” href=”/manifest.json”>
A manifest can specify an app name, icon, display mode and orientation:
In the run up to the “Mozlandia” work week in Portland, and in reflection of the last three years of the Firefox OS project, for a bit of fun I’ve reworked a Bob Dylan song to celebrate our incredible journey so far.
Here’s a video featuring some of my memories from the last three years, with Siobhan (my fiancée) and me singing the song at you! There are even lyrics so you can sing along
What is a web app? What is the difference between a web app and a web site? What is the difference between a web app and a non-web app?
The presence of web browser-like user interface elements like a URL bar and navigation controls are likely to make a user feel like they’re using a web site rather than an app for example, whereas content which appears to run independently of the browser feels more like an app. Apps are generally assumed to have at least limited functionality without an Internet connection and tend to have the concept of residing in a self-contained way on the local device after being “installed”, rather than being navigated to somewhere on the Internet.
From a technical point of view there is in fact usually very little difference between a web site and a web app. Different platforms currently deal with the concept of “web apps” in all sorts of different, incompatible ways, but very often the main difference between a web site and web app is simply the presence of an “app manifest”. The app manifest is a file containing a collection of metadata which is used when “installing” the app to create an icon on a homescreen or launcher.
At the moment pretty much every platform has its own proprietary app manifest format, but the W3C has the beginnings of a proposed specification for a standard “Manifest for web applications” which is starting to get traction with multiple browser vendors.
Web Manifest – Describing an App
Below is an example of a web app manifest following the proposed standard format.
The manifest file is referenced inside the HTML of a web page using a link relation. This is cool because with this approach a web app doesn’t have to be distributed through a centrally controlled app store, it can be discovered and installed from any web page.
As you can see from the example, these basic pieces of metadata which describe things like a name, an icon and a start URL are not that interesting in themselves because these things can already be expressed in HTML in a web standard way. But there are some other other proposed properties which could be much more interesting.
Display Modes – Breaking out of the Browser
We said above that one thing that makes a web app feel more app like is when it runs outside of the browser, without common browser UI elements like the URL bar and navigation controls. The proposed “display” property of the manifest allows authors of web content which is designed to function without the need for these UI elements to express that they want their content to run outside of the browser.
The proposed display modes are “fullscreen”, “standalone”, “minimal-ui” and “browser”. The “browser” display mode opens the content in the user agent’s conventional method (e.g. a browser tab), but all of the other display modes open the content separate from the browser, with varying levels of browser UI.
There’s also a proposed “orientation” property which allows the content author to specify the default orientation (i.e. portrait/landscape) of their content.
App Scope – A Slice of the Web
In order for a web app to be treated separately from the rest of the web, we need to be able to define which parts of the web are part of the app, and which are not. The proposed “scope” property of the manifest defines the URL scope to which the manifest applies.
By default the scope of a web app is anything from the same origin as its manifest, but a single origin can also be sliced up into multiple apps or into app and non-app content.
Below is an example of a web app manifest with a defined scope.
From the user’s point of view they can browse around the web, seamlessly navigating between web apps and web sites until they come across something they want to keep on their device and use often. They can then slice off that part of the web by “bookmarking” or “installing” it on their device to create an icon on their homescreen or launcher. From that point on, that slice of the web will be treated separately from the browser in its own “app”.
Without a defined scope, a web app is just a web page opened in a browser window which can then be navigated to any URL. If that window doesn’t have any browser-like navigation controls or a URL bar then the user can get stranded at a dead on the web with no way to go back, or worse still can be fooled into thinking that a web page they thought was part of a web app they trust is actually from another, malicious, origin.
The web browser is like a catch-all app for browsing all of the parts of the web which the user hasn’t sliced off to use as a standalone app. Once a web app is registered with the user agent as managing a defined slice of the web, the user can seamlessly link into and out of installed web apps and the rest of the web as they please.
Service Workers – Going Offline
We said above that another characteristic users often associate with “apps” is their ability to work offline, in the absence of a connection to the Internet. This is historically something the web has done pretty badly at. AppCache was a proposed standard intended for this purpose, but there are many common problems and limitations of that technology which make it difficult or impractical to use in many cases.
A new, much more versatile, proposed standard is called Service Workers. Service Workers allow a script to be registered as managing a slice of the web, even whilst offline, by intercepting HTTP requests to URLs within a specified scope. A Service Worker can keep an offline cache of web resources and decide when to use the offline version and when to fetch a new version over the Internet.
The programmable nature of Service Workers make them an extremely versatile tool in adding app-like capabilities to web content and getting rid of the notion that using the web requires a persistent connection to the Internet. Service Workers have lots of support from multiple browser vendors and you can expect to see them coming to life soon.
The proposed “service_worker” property of the manifest allows a content author to define a Service Worker which should be registered with a specified URL scope when a web app is installed or bookmarked on a device. That means that in the process of installing a web app, an offline cache of web resources can be populated and other installation steps can take place.
Below is our example web app manifest with a Service Worker defined.
The reason this approach is commonly taken is that it allows operating system developers and content authors to side-step some of the current shortcomings of the web platform. Packaging all of the resources of an app into a single file which can be downloaded and installed on a device is the simplest way to solve the offline problem. It also has the convenience that the contents of that package can easily be reviewed and cryptographically signed by a trusted party in order to safely give the app privileged access to system functions which would currently be unsafe to expose to the web.
Unfortunately the packaged app approach misses out on many of the biggest benefits of the web, like its universal and inter-linked nature. You can’t hyperlink into a packaged app, and providing an updated version of the app requires a completely different mechanism to that of web content.
We have seen above how Service Workers hold some promise in finally solving the offline problem, but packages as a concept may still have some value on the web. The proposed “Packaging on the Web” specification is exploring ways to take advantage of some of the benefits of packages, whilst retaining all the benefits of URLs and the web.
This specification does not explore a new security model for exposing more privileged APIs to the web however, which in my view is the single biggest unsolved problem we now have left on the web as a platform.
In conclusion, a look at some of the latest emerging web standards tells us that the answer to the question “what is a web app?” is that a web app is simply a slice of the web which can be used separately from the browser.
With that in mind, web authors should design their content to work just as well inside and outside the browser and just as well offline as online.
Packaged apps are not web apps and are always a platform-specific solution. They should only be considered as a last resort for apps which need access to privileged functionality that can’t yet be safely exposed to the web. New web technologies will help negate the need for packages for offline functionality, but packages as a concept may still have a role on the web. A security model suitable for exposing more privileged functionality to the web is one of the last remaining unsolved challenges for the web as a platform.
The web is the biggest ecosystem of content that exists, far bigger than any proprietary walled garden of curated content. Lots of cool stuff is possible using web technologies to build experiences which users would consider “app like”, but creating a great user experience on the web doesn’t require replicating all of the other trappings of proprietary apps. The web has lots of unique benefits over other app platforms and is unrivalled in its scale, ubiquity, openness and diversity.
It’s important that as we invent cool new web technologies we remember to agree on standards for them which work cross-platform, so that we don’t miss out on these unique benefits.
As soon as the Boot to Gecko (B2G) project was announced in July 2011 I knew it something I wanted to contribute to. I’d already been working on the idea of a browser based OS for a while but it seemed Mozilla had the people, the technology and the influence to build something truly disruptive.
At the time Mozilla weren’t actively recruiting people to work on B2G, the team still only consisted of the four co-founders and the project was little more than an empty GitHub repository. But I got in touch the day after the announcement and after conversations with Chris, Andreas and Mike over Skype and a brief visit to Silicon Valley, I somehow managed to convince them to take me on (initially as a contractor) so I could work on the project full time.
A Web Browser Built from Web Technologies
On my first day Chris Jones told me “The next, highest-priority project is a very basic web browser, just a URL bar and back button basically.”
Chris and his bitesize browser, Taipei, December 2011
The team was creating a prototype smartphone user interface codenamed “Gaia”, built entirely with web technologies. Partly to prove it could be done, but partly to find the holes in the web platform that made it difficult and fill those holes with new Web APIs. I was asked to work on the first prototypes of a browser app, a camera app and a gallery app to help find some of those holes.
You might wonder why a browser-based OS needs a browser app at all, but the thinking for this prototype was that if other smartphone platforms had a browser app, then B2G would need one too.
In the beginning, there was an <iframe>
It all started with a humble iframe, a text input for the URL bar and a go button, in fact you can see the first commit here. When you clicked the go button, it set the src attribute of the iframe to the contents of the text input, which caused the iframe to load the web page at that URL.
First commit, November 2011
Another problem we came across quite quickly was that many web authors will go to great lengths to prevent their web site being loaded inside an iframe in order to prevent phishing attacks. A web server can send an X-Frame-Options HTTP response header instructing a user agent to simply not render the content, and there are also a variety of techniques for “framebusting” where a web site will actively try to break out of an iframe and load itself in the parent frame instead.
It was quickly obvious that we weren’t going to get very far building a web browser using web technologies without evolving the web technologies themselves.
The Browser API
I met Justin Lebar at the first B2G work week in Taipei in December 2011. He was tasked with modifying Gecko to make the browser app on Boot to Gecko possible. To me Gecko was (and largely still is) a giant black box of magic spells which take the code I write and turn it into dancing images on the screen. I needed a wizard who had a grasp on some of these spells, including a particularly strong spell called Docshell which only the most practised of wizards dare peer into.
Justin at the first B2G Work Week in Taipei, December 2011
When I told Justin what I needed he made the kinds of sounds a mechanic makes when you take your car in for what you think is a simple problem but turns out costing the price of a new car. Justin had a better idea than I did as to what was needed, but I don’t think either of us realised the full scale of the task at hand.
With the adding of a simple boolean “mozbrowser” attribute to the HTML iframe element in Gecko, the Browser API was born. I tried adding features to the browser app and every time I found something that wasn’t possible with current web technologies, I went back to Justin to get him to cast a new magic spell.
There were easier approaches we could have taken to build the browser app. We could have added a mechanism to allow the browser to inject scripts into the iframe and communicate freely with the content inside, but we wanted to provide a safe API which anyone could use to build their own browser app and this approach would be too risky. So instead we built an explicit privileged API into the DOM to create a new class of iframe which could one day become a new standard HTML tag.
Keeping the Web Contained
The first thing we did was to try to trick web pages loaded inside an iframe into thinking they were not in fact inside an iframe. At first we had a crude solution which just ignored X-Frame-Options headers for iframes in whitelisted domains that had the mozbrowser attribute. That’s when we discovered that some web sites are quite clever at busting out of iframes. In the end we had to take other measures like making sure window.top pointed at the iframe rather than its parent so a web site couldn’t detect that it had a parent, and eventually also run every browser tab in its own system process to completely isolate them from each other.
Once we had the animal that is the web contained, we needed to poke a few air holes to let it breathe. There’s some information we need to let out of the iframe in the form of events: when the location, title or icon of a web page changes (locationchange, titlechange and iconchange); when a page starts and finishes loading (loadstart, loadend) and when the security characteristics of the currently loaded page changes (securitychange). This all allows us to keep the address bar and title bar up to date and show a progress indicator.
The browser app needs to be able to navigate the iframe by telling it to goBack(), goForward(), stop() and reload(). We also need to be able to explicitly ask for information like characteristics of the session history (getCanGoBack(), getCanGoForward()) to determine which navigation buttons to display.
With these basics in place it was possible to build a simple functional browser app.
The Gaia project’s first UX designer was Josh Carpenter. At an intensive work week in Paris the week before Mobile World Congress in February 2012, Josh created UI mockups for all the basic features of a smartphone, including a simple browser, and we built a prototype to those designs.
Josh and me plotting over a beer in Paris.
The prototype browser app could navigate web content, keep it contained and display basic information about the content being viewed. This would be the version demonstrated at MWC in Barcelona that year.
Simple browser demo for Mobile World Congress, February 2012
Building a Team
At a work week in Qualcomm’s offices in San Diego in May 2012 I was able to give a demo of a slightly more advanced basic browser web app running inside Firefox on the desktop. But it was still very basic. We needed a team to start building something good enough that we could ship it on real devices.
“Browser Inception”, San Diego May 2012
San Diego was also where I first met Dale Harvey, a brave Scotsman who came on board to help with Gaia. His first port of call was to help out with the browser app.
Dale Getting on Board in San Diego, May 2012
One of the first things Dale worked on was creating multiple tabs in the browser and even adding a screenshotting spell to the Browser API to show thumbnails of browser tabs (I told you he was brave).
By this time we had also started to borrow Larissa Co, a brilliant designer from the Firefox team, to work on the interaction design and Patryk Adamczyk, formerly of RIM, to work on the visual design for the browser on B2G. That was when it started to look more like a Firefox browser.
Early UI Mockup, July 2012
Things that Pop Up
Web pages like to make things pop up. For a start they like to alert(), prompt() or confirm() things with you. Sometimes they like to open() a new browser window (and close() them again), open a link in a _blank window, ask you for a password, ask for your permission to do something, ask you to select an option from a menu, open a context menu or confirm re-sending the contents of a form.
An alert(), version 1.0
All of this required new events in the Browser API, which meant more spells for Justin to cast.
Scroll, Pan and Zoom
Moving around web pages on web devices works a little differently from on the desktop. Rather than scroll bars or a scroll wheel on a mouse it uses touch input and a system called Asynchronous Pan and Zoom to allow the user to pan around a web page by dragging it and scrolling it using “kinetic scrolling” which feels like it has some physics to it.
One of the trickier interactions to get right was that we wanted the address bar to hide as you scrolled down the page in order to make more room for content, then show again when you scroll back to the top of the page.
This required adding asyncscroll events which tapped directly into the Asynchronous Pan and Zoom code so that the browser knew not only when the user directly manipulated the page, but how much it scrolled based on physics, asynchronously from the user’s interaction.
One of the most loved features of Firefox is the “Awesomebar”, a combined address bar, search bar (and on mobile, title bar) which lets you quickly get to the content you’re looking for. You type a few characters and immediately start to see matching web pages from your browsing history, ranked by a “frecency” algorithm.
On the desktop and on Android all of this data is stored in the “Places” database as part of privileged “chrome” code. In order to implement this feature in B2G we would need to use the local storage capabilities of the web, and for that we chose IndexedDB. We built a Places database in IndexedDB which would store all of the “places” a user visits on the web including their URL, title and icon, and store all the times the user visited that page. It would also be used to store the users bookmarks and rank top sites by “frecency”.
Awesomebar, version 1.0
As you browse around the web Gecko also stores a bunch of data about the places you’ve been. That can be cookies, offline pages, localStorage, IndexedDB databases and all sorts of other bits of data. Firefox browsers provide a way for you to clear all of this data, so methods needed to be added to the Browser API to allow this data to be cleared from the browser settings in B2G.
Browser settings, version 1.0
Sometimes web pages crash the browser. In B2G every web app and every browser tab runs in its own system process so that should the worst happen, it will only cause that one window/tab to crash. In fact, due to the memory constraints of the low-end smartphones B2G would initially target, sometimes the system will intentionally kill a background app or browser tab in order to conserve memory. The browser app needs to be informed when this happens and needs to be able to recover seamlessly so that in most cases the user doesn’t even realise a process was killed. Events were added to the Browser API for this purpose.
Crashed tab, version 1.0
Talking to Other Apps
Common use cases of a mobile browser are for the user to want to share a URL using another app like a social networking tool, or for another app to want to view a URL using the browser.
B2G implemented Web Activities for this purpose, to add a capability to the web for apps to interact with each other, but in an app-agnostic way. So for example the user can click on a share button in the browser app and B2G will fire a “share URL” Web Activity which can then be handled by any installed app which has registered to handle that type of Web Activity.
Share Web Activity, version 1.2
Despite the fact that B2G and Gaia are built on the web, it is a requirement that all of the built-in Gaia apps should be able to function offline, when an Internet connection is unavailable or patchy, so that the user can still make phone calls, take photos and listen to music etc.. At first we started to use AppCache for this purpose, which was the web’s first attempt at making web apps work offline. Unfortunately we soon ran into many of the common problems and limitations of that technology and found it didn’t fulfill all of our requirements.
In order to ship version 1.0 of B2G on time, we were forced to implement “packaged apps” to fulfill all of the offline and security requirements for built-in Gaia apps. Packaged apps solved our problems but they are not truly web apps because they don’t have a real URL on the Internet, and attempts to standardise them didn’t get much traction. Packaged apps were intended very much as a temporary solution and we are working hard at adding new capabilities like ServiceWorkers,standardised hosted packages and manifests to the web so that eventually proprietary packaged apps won’t be necessary for a full offline experience.
Offline, version 1.4
Spit and Polish
Finally we applied a good deal of spit and polish to the browser app UI to make it clean and fluid to use, making full use of hardware-accelerated CSS animations, and a sprinkling of Firefoxy interaction and visual design to make the youngest member of the Firefox browser family feel consistent with its brothers and sisters on other platforms.
At an epic work week in Berlin in January 2013 hosted by Deutsche Telekom the whole B2G team, including engineers from multiple competing mobile networks and device manufacturers, got together with the common cause of shipping B2G 1.0, in time to demo at Mobile World Congress in Barcelona in February. The team sprinted towards this goal by fixing an incredible 200 bugs in one week.
Version 1.0 Team, Berlin Work Week, January 2013
In the last few minutes of the week Andreas Gal excitedly declared “Zarro Gaia Boogs”, signifying version 1.0 of Gaia was complete, with the rest of B2G to shortly follow over the weekend. Within around 18 months a dedicated team spanning multiple organisations had come together working entirely in the open to turn an empty GitHub repository into a fully functioning mobile operating system which would later ship on real devices as Firefox OS 1.0.1.
Zarro Gaia Boogs, January 2013
Browser app v1.0
So having attended Mobile World Congress 2012 with a prototype and a promise to deliver commercial devices into the market, we were able to return in 2013 having delivered on that promise by fully launching the “Firefox OS” brand with multiple devices on multiple mobile networks with a launch that really stole the show at the biggest mobile conference in the world. Firefox OS had arrived.
Mobile World Congress, Barcelona, February 2013
Firefox OS 1.1 quickly followed and by the time we started working on version 1.2 the project had grown significantly. We re-organised into autonomous agile teams focused on product areas, the browser app being one. That meant we now had a dedicated team with designers, engineers, a test engineer, a product manager and a project manager.
The browser team, London work week, July 2013
Firefox OS moved to a rapid release “train model” of development like Firefox, where a new version is delivered every 12 weeks. We quickly added new features and worked on improving performance to get the best out of the low end hardware we were shipping on in emerging markets.
Browser app v1.4
Version 1.0 of Firefox OS was very much about proving that we could build what already exists on other smartphones, but entirely using open web technologies. That included a browser app.
Once we’d proved that was possible and put real devices on shelves in the market it was time to figure out what would differentiate Firefox OS as a product going forward. We wanted to build something that doesn’t just imitate what’s already been done, but which plays to the unique strengths of the web to build something that’s true to Mozilla’s DNA, is the best way to experience the web, and is the platform that HTML5 deserves.
Below is a mockup I created right back towards the start of the project at the end of 2011, before we even had a UX team. I mentioned earlier that the Awesomebar is a core part of the Firefox experience in Firefox browsers. My proposal back then was to build a system-wide Awesomebar which could search the whole device, including your apps and their contents, and be accessible from anywhere in the OS.
Very early mockup of a system-wide Awesomebar, December 2011
At the time, this was considered a little too radical for version 1.0 and our focus really needed to be on innovating in the web technology needed to build a mobile OS, not necessarily the UX. We would instead take a more conservative approach to the user interface design and build a browser app a lot like the one we’d built for Android.
In practice that meant that we in fact built two browsers in Firefox OS. One was the browser app which managed the world of “web sites” and the other was the window manager in the system app which managed the world of “web apps” .
In reality on the web there isn’t so much of a distinction between web apps and web sites – each exists on a long continuum of user experience with a very blurry boundary in the middle.
Gordon’s Rocketbar Prototype, March 2013
Gordon and I started to meet weekly to discuss the concept he had by then codenamed “Rocketbar”, but it was a bit of a side project with a few interested people.
In April 2013 the UX team had a summit in London where they got together to discuss future directions for the user experience of Firefox OS. I was lucky enough to be invited along to not only observe but participate in this process, Josh being keen to maintain a close collaboration between Design and Engineering.
We brainstormed around what was unique about the experience of the web and how we might create a unique user experience which played to those strengths. A big focus was on “flow”, the way that we can meander through the web by following hyperlinks. The web isn’t a world of monolithic apps with clear boundaries between them, it is an experience of surfing from one web site to another, flowing through content.
Brainstorming session, London, April 2013
In the coming weeks the UX team would create some early designs for a concept (eventually codenamed “Haida”) which would blur the lines between web apps and web sites and create a unique user experience which flows like the web does. This would eventually include not only the “Rocketbar”, which would be accessible across the whole OS and seamlessly adapt to different types of web content, but also “sheets”, which would split single page web apps into multiple pages which you could swipe through with intuitive edge gestures. It would also eventually include a content model based around live apps which you can surf to, use, and then bookmark if you choose to, rather than monolithic apps which you have to install from a central app store before you can use them.
In June 2013 a small group of designers and engineers met in Paris to develop a throwaway prototype of Haida, to rapidly iterate on some of the more radical concepts and put them through user testing.
Haida Prototyping, Paris, June 2013
Josh and Gordon working in a highly co-ordinated fashion, Paris, June 2013
Wizards at work, Paris, June 2013
2.x and the Future
Fast forward to the present and the browser team has been merged into the “Systems Front End” team. The results of the Haida prototyping and user testing are slowly starting to make their way into the main Firefox OS product. It won’t happen all at once, but it will happen in small pieces as we iterate and learn.
In version 2.0 of Firefox OS the homescreen search feature from 1.x will be replaced with a new search experience developed in conjunction with a new homescreen, implemented by Kevin Grandon, which will lay the foundations for “Rocketbar”. In version 2.1 our intention is to completely merge the browser app into the system app so that browser tabs become “sheets” alongside apps in the task manager and the “Rocketbar” is accessible from anywhere in the OS. The Rocketbar will adapt to different types of web content and shrink down into the status bar when not in use. Edge gestures will allow you to swipe between web apps and browser windows and eventually apps will be able to spawn multiple sheets.
UI Mockups of Rocketbar in expanded and collapsed state, July 2014
Version 1.x of Firefox OS was built with web technologies but still has quite a similar user experience to other mobile platforms when it comes to installing and using apps, and browsing the web. Going forward I think you can expect to see the DNA of the web come through into the user interface with a unified experience which breaks down the barriers between web apps and web sites, allowing you to freely flow between the two.
Firefox OS is an open source project developed completely in the open. If you’re interested in contributing to Gaia, take a look at the “Developing Gaia” page on MDN. If you’re interested in creating your own HTML5 app to run on Firefox OS take a look at the “App Center“.
I’m trying to build a Raspberry Pi powered robot based on the DRDs from Farscape, I thought I’d blog my progress.
DRDs or “Diagnostic Repair Drones” are robots from the cult science fiction series Farscape. They carry out various functions aboard a leviathian (a species of living biomechanoid spaceship) including repairing and maintaining the ship. They’re ovoid in shape and they have two moving eye stalks and all sorts of tools like a robotic claw and a plasma welder.
Here’s some video footage from the series to give you an idea of what these little guys get up to:
The original DRDs were designed and built by the extremely talented folks at the Jim Henson Creature Shop in London (yes Jim Henson as in the Muppets!). They built lots of different variations of the robot over the years to be used in shooting different scenes for the show, but to my knowledge they’ve never released any designs.
I assumed I was going to have to painstakingly design a 3D computer model of one based on frame grabs from my DVDs of the series. I then planned to track down someone with a CNC router and a vacuum forming machine and persuade them to let me use them. Either that or find someone with an industrial sized 3D printer!
Luckily I came across a special effects company in the US who sells a kit to build a model of a DRD. The model is made from hollow cast fiberglass and resin and comes with ribbed plastic for the eye stalks, eye pieces with clear lenses, two parts of a claw and some colourful wires to make it look the part.
The kit isn’t perfect. The size, shape and proportions aren’t quite right and the finish is a bit rough but it’s good enough for my purposes. The part I’m really interested in is the robotics so I’m grateful that someone has already done the work for me on the basic shell.
The web site provides video tutorials on how to build the model and then how to put LEDs in the eyes and mount an remote controlled car underneath to make it move about in a bit of a crude fashion.
We can be a bit more sophisticated than that.
The Raspberry Pi is a credit-card sized computer developed in the UK by the not-for-profit Raspberry Pi Foundation to promote the teaching of programming in schools. It’s a single-board computer with a 700Mhz ARM processor and 512MB RAM, boots off an SD card and costs only around £30.
This is my Raspberry Pi:
The Gertboard is an expansion board which attaches to the Raspberry Pi via its GPIO pins and helps when experimenting with interfacing the Pi with the outside world. It comes with an Arduino compatible AVR microcontroller, analogue to digital converters, digital to analogue converters, a motor controller, push buttons, LEDs and much more.
Booting the Pi
The Raspberry Pi can boot Linux from an SD card and the most popular distribution is Raspbian which is a Debian-derivative. You can download an image and flash it to an SD card, or even buy an SD card with it already loaded.
To boot the Raspberry Pi all you need to do is insert your Raspbian SD card, plug it into a TV via either the HDMI port or the composite video port and power it up by plugging it into a Micro USB phone charger.
Here’s my Raspberry Pi booted and plugged into an old CRT TV:
Logging In Remotely
It’s cool that I can plug the Raspberry Pi into a TV, but I don’t want to be squinting at an old portable TV or sitting in the lounge next to my big flatscreen TV all the time I’m programming the robot, so I want to be able to log in remotely. Also, my plan is to build a web interface to control the robot over WiFi, so it’s going to need to connect to a network at some point.
First I plugged a USB keyboard into the Raspberry Pi and an ethernet cable to connect it to my network. The SSH daemon is already started by default, but I wanted to set a static IP address so that I always knew what to log in to.
I logged into the Raspberry Pi locally (the default username is pi and the password is raspberry) and edited the network configuration using the vi text editor.
$ sudo vi /etc/network/interfaces
I provided the following configuration to assign a static IP address of 192.168.1.42 on my local network:
iface lo inet loopback
iface eth0 inet static
Then restart the network interface with:
$ sudo ifdown -a
$ sudo ifup -a
Then check that I’m connected to the network, and the Internet by pinging Google.
$ ping google.com
I see that I’m successfully connected, so I can now log into the Raspberry Pi remotely using its new static IP.
From my desktop Linux box I type:
$ ssh firstname.lastname@example.org
type in the password “raspberry”, and voilà! I’m logged in.
I hope you weren’t expecting to see a finished robot! There’s a very long way to go yet.
If you desperately wanted to see a finished robot, here’s a picture of the last one I worked on, a line following robot we built at university powered by a PIC microcontroller.
Next I want to start playing around with the Gertboard and and make LEDs blink on and off from Python.
Through Krellian I will be able to continue to lead the Webian project, and I will also be taking up a new contract with the Mozilla Corporation to work with them on Boot to Gecko (B2G).
Like me and the other members of the Webian community, Mozilla believes that the open web can displace proprietary, single-vendor stacks for application development. The B2G project will include prototype APIs for exposing device and OS capabilities to web content, a privilege model to safely expose these new capabilities, a complete "low-level substrate" for Android-compatible devices and a collection of web apps to prioritise and prove the power of the platform.
Benefits to Webian
The potential benefits for Webian are enormous. Webian Shell was already hitting limitations of what is currently possible with Mozilla Chromeless and this new work on the core Mozilla platform promises to make many more of Webian's goals possible. While B2G initially focuses on the mobile space, Webian can focus on nettop and netbook form factors and perhaps eventually the two projects could even converge.
Sponsorship from Krellian will provide the ongoing resources necessary for running the Webian project and ensure that it remains free and open source.
I'm excited about this new chapter in Webian's story and believe more strongly than ever in the future of the open web.
The best thing about web apps is that they can run on any platform. This is because they use a set of open standards which are universally supported. The latest incarnations of these standards like HTML5 and CSS3 are more powerful than ever and it seems inevitable to me that the web will eventually win out as the open, universal app platform.
The current generation of popular desktop operating systems like Windows 7, OS X and Linux and mobile operating systems like iOS, Android and Windows Phone treat web apps as second class citizens, inferior to their native platform-specific counterparts. But that's all set to change.
Microsoft Windows 8, Google Chrome OS, The Linux Foundation's Tizen and Mozilla B2G will all join Webian in putting web apps centre stage, as the preferred category of app on their respective platforms.
Thanks to HTML5 and related standards web apps are already pretty powerful, but there is still some way to go before web apps can do everything that native apps can do. On mobile platforms new APIs are needed for access to telephony, messaging and cameras for example, and some kind of "app store" style distribution and installation mechanism is likely to be needed to provide a more app-like experience. "Installing" a web app is a little like bookmarking a web site and can provide an opportunity for the app to ask the user for all the permissions it needs up-front, download any resources which will be needed for offline use and perhaps add an shortcut icon to an app list.
Because there aren't currently established standards for all of these features, each platform is intially rolling its own approach. Apps for Chrome OS can only be found via Google's own "Chrome Web Store", you have to have a Google Account to install them and many of the apps found there will only work in Chrome. Windows 8 will have a set of APIs for its new Metro user interface, but apps written for Metro won't necessarily work on other platforms. We don't yet know much about Tizen, but previous mobile platforms like HP's WebOS and the MeeGo Web Runtime have also had platform-specific APIs.
An Open Approach
Mozilla is also working on a new set of web APIs, but in line with their mission of promoting openness on the web they're doing so in a more open way. You can see the progress of their APIs for telephony, messaging (SMS), battery, visibility, settings, power management and contacts on Bugzilla. These new APIs on Mozilla's core platform are the first step towards a new mobile OS from Mozilla called B2G or "Boot to Gecko". B2G will have a highly customisable UI built entirely with web technologies and will allow for the installation of Open Web Apps, as proposed by Mozilla Labs. With Open Web Apps, anyone can run their own app store or directory and the aim is to create cross-platform installable web apps.
Mozilla is also collaborating with Google on "Web Activities" or "Web Intents" which could provide a standard mechanism for apps to call other apps for certain functions. This collaboration is a promising sign because Mozilla can not define standards on their own and will need to co-operate with other browser & OS vendors.
For example, W3C Widgets provide a mechanism for installing a "widget" locally on a device by packaging up all of its resources in a zip file, including an XML manifest file. The specification was used for widgets in the Opera browser but when the MeeGo Web Runtime tried to use it as a method of installing fully fledged web apps it didn't work out very well.
Mozilla and Google are taking a different approach to installable web apps, both choosing to use a JSON manifest file and rely to a large extent on HTML5's offline capabilities for caching and updating offline resources. Even these two similar implementations have subtle differences which make them incompatible with each other so there's a real need for standardisation in this area if we're to have cross-platform installable web apps.
Hosted vs. Packaged
One slightly odd trend in the standardisation of "web apps" is the tendency to focus on "packaging" apps rather than "hosting" them. This seems to be an attempt to emulate the way mobile and desktop apps currently work by bundling up all of the app's resource in a "package" which is downloaded and installed locally. The whole package then needs to be updated every time any of the resources change. This seems odd to me because this isn't the way the web usually works and it doesn't seem very web-like at all.
The fact that an individual resource can be identified by a URI is what makes it a part of the web, so to package resources up in one big bundle to be downloaded and used locally seems to break the web paradigm and therefore these apps are not "web apps" in my view, just apps downloaded over the web. This is how W3C widgets work and it's also how Chrome "packaged" apps work and seems to be the direction of the Wholesale Apps Community, the EU funded Webinos project and the Native Web Apps community on the W3C web site.
It seems that web apps will be the preferred category of apps for a whole new generation of operating systems, but in order for these web apps to remain cross-platform new open standards must be agreed upon for APIs which give web apps all the power of native apps.
A mechanism for the discovery, installation and updating of hosted installable web apps and a trust framework which allows for secure open distribution is also something which I think desperately needs standardisation if the vision of a universal open web app platform is to be realised.
(Reposted from the Webian blog) Webian Shell is a full screen web browser for devices that don't need a desktop. As of today you can download a prototype which you can try for yourself.
A Graphical Shell for the Web
If you’re anything like me then you’ll find that most of the stuff you do on your PC these days happens in a web browser and the desktop environment you used to depend on is now just getting in your way. Modern web applications can do just about everything traditional desktop applications can do (like flashy graphics, sound, video and offline storage) so often all you really need is the browser. The idea of the Webian Shell project is to replace your computer’s interface with something much simpler, which treats web applications as first class citizens and does away with all the un-necessary clutter.
These are some of my ideas for future directions to take the prototype, but I’d love to hear your ideas.
Home screens – instead of a desktop, Shell could have multiple home screens like you see on mobile devices, but containing web widgets and icons for “installed” web apps. See Mozilla’s Open Web Apps project for some inspiration.
Hardware controls – any graphical shell for a hardware device needs the ability to monitor and control hardware for things like sound, network, GPS, battery etc.
Zoomable tiled window manager – zoom out to a view of all currently loaded web pages (like Panorama in Firefox or Expose in OS X), then zoom in to the page you want.
Some Google Chrome engineers uploaded a video they called a "Chromercial" to promote Chrome OS, much to the marketing department's dismay! They asked for video responses where you give your own take on Chrome OS.
I originally created the artwork I used in this video for my "Graphical Shell for the Web" design concept which basically portrays the same idea as Chrome OS (but was published before Chrome OS was released!)
I created the artwork using Inkscape and put the video together using PiTiVi. I used Creative Commons licensed music and sound effects and licensed the video itself under a Creative Commons Non-commercial Sharealike license so you can remix & share it as much as you like. You can download the source png slides and sound files (along with the completed video in ogg format) here (I also have the SVG source of the images if anyone wants it).
When I first heard about Google's and then Mozilla's concepts of "installable web apps" I was a little dubious. Surely the whole benefit of web applications is that they don't need to be installed!
But if you think of installable web apps as a convergence of the "app store" and "cloud computing" models (arguably the two most significant trends in the software industry today) it actually makes quite a lot of sense.
The diversity of different software platforms available for smartphones, tablets, smart TVs and PCs is great for encouraging competition and driving innovation – but it also causes problems for both app developers and consumers. An app written for Apple's iOS for example will not run on Android, Blackberry OS, WebOS, Meego, Bada, Symbian or any other competing platform. For consumers this is confusing and frustrating because unless they allow themselves to be locked in to a single vendor they may have to re-purchase different apps for different devices. For app developers it's even more frustrating because they're faced with a bewildering array of platforms to target their software at.
Web applications have different problems with limited discoverability and access to hardware. To a web browser web applications are no different to any other web site which means that the apps can easily get lost in the ocean of content on the web. Web apps also traditionally have very limited default access to client-side hardware for security reasons which limits their capabilities when compared to native applications.
Installable web apps can combine the best parts of web applications (the benefits of cloud computing and accessiblity from any device) with the best parts of the app store model (easy distribution and discovery of software which has full access to the device's hardware). This new breed of apps could leverage web technologies to reach the whole range of competing platforms with a single application which can be distributed via a range of competing app stores. By "installing" a web application the user can pre-approve access to local hardware like offline storage, accelerated graphics and geolocation to allow the app to make the most of the hardware it's running on and the user can keep a local repository of all the applications they use.
Both developers and consumers only have to worry about one kind of application running on one open platform (the web), but still have a choice of hardware and app stores. In this model the only people who lose out are the corporations who seek to lock consumers into their own products and services.
It could take a long time for installable web apps to become the predominant model of software distribution, but the idea of converging cloud computing with the app store model makes the idea very appealing and might go some way to explaining why Google currently has both the Android and Chrome OS operating systems in development. Each OS is approaching the convergence from a different direction whilst maximising on the market potential of what can already be done today.
I hope that standards for "installable web apps" can be agreed upon sooner rather than later so that if this model does become a growing trend, the new breed of apps will not suffer from the same fragmentation as other platforms do today.