A New Future for the WebThings IoT Platform


Originally posted on Medium.

After four years of incubation at Mozilla, Krellian is proud to become the new commercial sponsor of WebThings, an open platform for monitoring and controlling devices over the web.

Today we are announcing the release of WebThings Gateway 1.0 and setting out a vision for the future of the WebThings project.

WebThings

WebThings is an open source implementation of emerging W3C Web of Things standards and consists of three main components:

  • WebThings Gateway — a software distribution for smart home gateways which enables users to directly monitor and control their home over the web, without a middleman
  • WebThings Framework — a collection of re-usable software components to help developers build their own web things which directly expose the Web Thing Protocol
  • WebThings Cloud — a collection of cloud services for securely deploying connected devices and remotely managing them over the internet

Flying the Nest

Following a company restructuring in August, Mozilla was looking for a new home for the WebThings community to continue their work.

Having co-founded the project whilst working at Mozilla, I joined discussions with two of my former colleagues Michael Stegeman and David Bryant about spinning out WebThings as an independent open source project. We worked with Mozilla on an agreement to transition the project to a new community-run home at webthings.io, and have spent the last three months working together on that transition.

WebThings Gateway 1.0

Today marks the public release of WebThings Gateway 1.0 and the formal transition of the WebThings platform to its new home at webthings.io. Going forward, Krellian will be sponsoring the new WebThings website and replacement cloud infrastructure, to continue to provide automatic software updates and a secure remote access service for WebThings gateways around the world.

You can read more about the 1.0 release and the transition of existing gateways to the new infrastructure on the Mozilla Hacks blog.

Krellian & WebThings

Krellian’s mission is to “extend the World Wide Web into physical spaces to make our built environment smarter, safer and more sustainable.” WebThings provides an ideal open source platform, built on web standards, to help achieve that mission.

In the short term Krellian will be leveraging the WebThings Cloud remote access service as part of our new digital signage platform. In the longer term we plan to explore other enterprise use cases for the WebThings platform, to help make buildings smarter, safer and more sustainable.

These commercial applications of WebThings will help provide revenue streams to support the long term sustainability of the open source project and allow it to continue to develop and grow.

The WebThings Community

Krellian highly values the thriving community who have supported the WebThings project over the last four years. From hackers and makers to educators and hobbyists, the community have been pivotal in building, testing and promoting WebThings around the world.

Amongst their achievements is the translation of WebThings Gateway into 34 spoken languages, the creation of over a hundred gateway add-ons and the building of countless DIY projects in a dozen different programming languages. Community members have contributed their time and effort to help build and promote WebThings and support other members in using it in thousands of private smart homes around the world.

We intend to support the community to continue with their great work, and have put in place an open governance structure to distribute decision making and foster leadership amongst the global WebThings community.

Future Roadmap

The following are some ideas about where to take the platform next, but we’d also very much like to hear from the community about what they would like to see from the project going forward.

W3C Compliance

WebThings has been developed in parallel with, and has contributed to, the standardisation of the Web of Things at the W3C. Since the last release of WebThings Gateway in April, the W3C Thing Description specification has reached “recommendation” status and is now an international standard.

We’d like to work towards making WebThings compliant with this standard, as there are still a remaining number of differences between the W3C and Mozilla specifications. In order to fill in the gaps between Mozilla’s Web Thing API and the W3C’s Thing Description standard, we plan to continue to lead work on standardising the Web Thing Protocol as a concrete protocol for communicating with devices over the web.

Production Gateway OS

The main WebThings Gateway software image is currently built on top of the Raspbian Linux distribution. This served the project well for its initial target of DIY smart home users, using the popular Raspberry Pi single board computer.

As the platform matures, we would like to explore a more production-quality IoT operating system like Ubuntu Core or Balena OS on which to base the WebThings Gateway distribution.

This will have the following benefits:

  1. A smaller footprint, reducing the minimum system requirements for running the gateway
  2. Enabling the targeting of a wider range of hardware for consumer and enterprise use cases
  3. Better security, through containerisation and automatic software updates for the underlying operating system

WebThings Controller

There was previously a project to build controller software for WebThings, to run on a controller device such as a smart speaker or smart display. The initial prototype was built on Android Things, but was discontinued when Google locked down the Android Things platform to specific OEMs and introduced restrictions on how it could be used.

Krellian would like to explore new controller software built on our open source Krellian Kiosk web runtime, which could allow for touch and voice input. This software would be designed so that it could either run on the same device as the gateway software, or on a separate controller device.

WebThings App

A native WebThings mobile app could act as a general purpose Web of Things client. This could potentially:

  1. Help to streamline the setup process of a WebThings Gateway
  2. Act as a client for native web things which don’t need a gateway
  3. Help with the standardisation process by providing a user friendly reference implementation of a Web of Things client

WebThings Cloud

Finally, we would like to explore expanding the WebThings Cloud offering. This could include an online dashboard for monitoring and controlling devices across multiple premises, and cloud to cloud integrations with other IoT platforms and voice assistants.


We’re excited about this new chapter in the WebThings story, and look forward to working closely with the community on our vision of a connected world where technology is seamlessly woven into the spaces around us and improves the lives of those who use it.

You can find out more about WebThings at its new home of webthings.io, follow @WebThingsIO on Twitter and sign up for the email newsletter to keep up to date with all the latest news.

Introducing Krellian — Interactive Digital Signage

Orignally posted on Medium.

For the last six weeks I’ve been enrolled in YCombinator’s Startup School, working towards the launch of a new technology startup.

Today I’m excited to introduce Krellian, a software platform for interactive digital signage, built on web standards.

If you walk around any large town or city today you’ll notice that you’re surrounded by screens. Digital billboards, information kiosks, self-service kiosks, interactive exhibits, digital menus and departure boards.

A lot of those screens are running outdated consumer-grade operating systems and proprietary content runtimes. They’re unreliable, inefficient and insecure. How many times have you seen a digital billboard that’s gone dark, departure boards with a Windows error message on the screen or even ATMs with a blue screen of death?

With Krellian, I am building a software platform for interactive digital signage, built on web standards. A simple, reliable web-based operating system and a secure cloud service for monitoring, controlling and deploying content to connected displays over the internet.

Krellian’s products will build on everything I have learnt over the last decade working on Webian, Firefox OS and Mozilla WebThings. I believe the web technologies I’ve helped standardise around installable web apps and the Web of Things could have enormous potential if applied to the digital signage market.

For example, a purpose-built operating system for displaying modern web content, which can be remotely managed over the internet, could significantly reduce content creation costs and technician call-out fees.

I’m also delighted that Krellian has been accepted onto the High Potential Startups programme, powered by the North East Local Enterprise Partnership. Over the next six months we’ll be working together to better understand customer needs, bring products to market and ultimately grow the business to create more high-tech jobs in the North East of England.

Are you interested in providing in-person digital experiences to your customers? Perhaps you work in advertising, hospitality, healthcare, museums, travel, retail or entertainment? Or do you sell digital signage to your own customers? I’d love to hear more about your needs and the problems you are trying to solve.

Register your interest today at krellian.com and follow Krellian on Twitter and Facebook.

The Legacy of Firefox OS

Originally posted on Medium.

In 2017 I wrote about the story of Firefox OS, from the birth of the B2G (Boot to Gecko) open source software project as an empty GitHub repository in 2011, to shipping as Firefox OS on five million devices worldwide, and then eventually being “transitioned” to the community in 2016.

In the two years or so since Mozilla announced the end of Firefox OS as a Mozilla-run project, the B2G source code has found its way into a surprising number of commercial products. In fact since Mozilla moved on from Firefox OS, its derivatives have shipped on an order of magnitude more devices than during its entire time under Mozilla’s leadership and it has gone on to form the basis of the third largest and fastest growing mobile operating system in the world.

In this blog post I take a look at the legacy of the B2G project, some of the devices you can buy today running B2G code, and other related efforts which still continue inside and outside Mozilla. I will cover KaiOS, Panasonic’s My Home Screen, Servonk, Webian, Astian OS, IceWolf OS and the latest in Mozilla Research & Development. I’ll then explain why I think there’s a strong argument for Mozilla to take a more active role in supporting downstream users of Gecko in future, and even the B2G codebase itself.

KaiOS

Probably the most notable fork of the B2G source code is KaiOS from KaiOS Technologies. It was started in 2016 with a team of 30 people and has since grown to have offices in France, Germany, Taiwan, India, Brazil, Hong Kong and mainland China in addition to its headquarters in the US.

KaiOS logo

KaiOS has a UI optimised for non-touch “smart feature phones” with physical keypads and small colour screens. That includes the hugely popular JioPhone in India, HMD’s nostalgic re-launch of the Nokia 8110, and the recently announced $7 WizPhone in Indonesia. With these devices KaiOS is very much continuing the Firefox OS vision of “connecting the next billion”, with low end devices aimed mainly at emerging markets.

KaiOS Technologies CEO Sebastien Codeville with KaiOS devices at CES 2019

JioPhone

The JioPhone has been extremely popular in India since its launch in July 2017, in part due to being given away for free, with a security deposit of ₹1500 ($21 USD) which can be refunded by returning the device to Jio stores after three years.

The JioPhone 2 was released in July 2018 at a price of ₹2,999 ($42 USD). It has a full QWERTY keyboard and has apps for Facebook, WhatsApp and YouTube.

JioPhone & JioPhone 2

Nokia 8110

The Nokia 8110 is available internationally, including here in the UK with a recommended retail price of £69.99 ($89) unlocked. It’s available from many popular retailers and mobile network providers including Carphone Warehouse, Currys, John Lewis, Amazon, Vodafone, EE, GiffGaff and BT.

Nokia 8110

The Nokia 8110 is a very neat little device with an intuitive UI reminiscent of Symbian, but with a modern twist. It’s surprising how much of the web actually works on such a small non-touch screen, though it’s a very fiddly experience if you’re used to a touch screen smartphone.

The presence of Google Assistant on feature phones at this price point is remarkable and it works surprisingly well. To implement Google Assistant for KaiOS, Google have created a custom HTML5 app packaged for the mozApp app runtime KaiOS inherited from B2G.

Nokia 8110 running Google Assistant

When I first bought the Nokia 8110 the bundled app store had a grand total of eight apps in it and there’s no way to install web apps from the browser. Since then it has grown a little, but only to about 30 apps and most of them are games. (The JioPhone comes with its own app store called JioStore and not all apps are available on all KaiOS devices, e.g. WhatsApp.)

WizPhone WP006

The WizPhone WP006 was recently announced in Indonesia in partnership with Google and Alfamart and can be purchased from vending machines for only IDR99,000 ($7).

WizPhone WP006

Other KaiOS Devices

Other devices running KaiOS include:

Rapid Growth

By the end of 2017 KaiOS had shipped on 30 million phones in the U.S., Canada, and India via partnerships with 20 carriers.

By June 2018 it had reached 40 million devices and Google made a $22m investment in the company. As part of that agreement KaiOS Technologies agreed to bundle Google services, including Google Assistant.

As of the end of 2018 it has reached over 50 million devices across North America, Asia, Europe, and Africa.

According to StatCounter, as of the end of 2018 KaiOS was the third largest mobile operating system in the world after Android and iOS, with 1.13% market share worldwide.

Worldwide mobile operating system market share, December 2018. Source: StatCounter

In India it has 4.55% market share, beating Apple’s iOS into second place.

India mobile operating system market share, December 2018. Source: StatCounter

Strategy Analytics are forecasting that KaiOS device shipments will grow 50% year on year in 2019, with more than 30% of those shipments coming from non-India markets such as Africa, South East Asia and Latin America. This, they say, will make it the “world’s fastest growing major platform”.

Panasonic My Home Screen

KaiOS is not the only fork of B2G. Panasonic was the first hardware vendor to create Firefox OS based smart TVs, which received great reviews for their user friendly design when they were released in 2015. When Mozilla stopped work on Firefox OS, Panasonic continued development of the software as My Home Screen 2.0 and later My Home Screen 3.0.

Panasonic 65″ OLED 4K Smart TV (TX-65FZ802B) running My Home Screen 3.0

As of 2019 most Panasonic smart TVs are still running this B2G fork. There are localised variations being sold running My Home Screen all over the world including in North America, Latin America, Europe, Middle East, Africa, Asia and Oceania. They range from the low end 24″ LED TX-24FS500B in the UK for £279 ($359) to the very high end 77″ OLED TH-77EZ1000H in Hong Kong for HK$139,000 ($17,700).

A notable exception is the US, where Panasonic recently stopped selling TVs due to difficult market conditions. But here in the UK you’ll come across these smart TVs when walking into most popular high street electronics retailers like John Lewis, Currys and Argos. Some listings on online stores even still refer to these TVs as “powered by Firefox OS”, despite the official change in branding.

My Home Screen 3.0 is probably a bit of a Frankenstein of web and native code under the hood at this point, with support for a wide range of live and on-demand video platforms, but the user interface has remained largely the same.

Panasonic 49″ 4K TV (TX-49FX750B) running My Home Screen 3.0

I have Panasonic TVs running both the Firefox OS branded and My Home Screen 3.0 branded software and while the browser has barely changed, app development has become much more locked down.

With the latest “HTML v3 SDK” (based on the “Hybrid broadcast broadband TV” standard), developer accounts are basically invitation only. I applied for an account and Panasonic first wanted to know what app I was developing so they could decide whether they wanted it in their store.

This is quite a contrast from the original Firefox OS TVs where anyone could enable developer mode and and anyone could submit an app to the Firefox Marketplace.

Still, with most of the major apps like Netflix, YouTube and Amazon Prime Video already supported, My Home Screen continues to provide a slick and compelling user experience for smart TVs.

In total Panasonic sells approximately 7 million TVs per year worldwide.

Servonk

In January 2018 the tech lead of KaiOS, Fabrice Desré (previously tech lead of B2G at Mozilla), started an open source project called Servonk. Servonk is kind of a “Boot to Servo” like the original Boot to Gecko project, but based on Mozilla’s experimental Servo rendering engine.

Servonk running in an emulator
Servonk running on an Android device

Servonk takes advantage of the cutting edge features of Servo, is largely built on the Rust programming language and roughly follows the slimmed down architecture we started to transition to towards the end of the B2G project.

Whilst a very cool open source project, Servonk is unlikely to ship on commercial devices any time soon, although there are some early discussions around shipping a version of the Necunos NC_1 (community edition) with B2G or Servonk pre-loaded.

Servo is still a very experimental browser engine which can’t render the majority of web pages on the web today. Inside Mozilla, work on Servo has recently mostly been focused on VR applications and moving parts of Servo over to Gecko as part of the Quantum project. Servo itself is currently still a research project.

According to the README, the main motivation of Servonk is to scratch a personal itch to build an alternative open source OS, and encourage contributions to Servo from outside Mozilla. If you join the (still surprisingly busy) #b2g channel on irc.mozilla.org, you’ll find that most recent discussions are around Servonk.

Webian

My own Webian project predated B2G but still continues to crawl along, learning from many of the lessons of Firefox OS and using the web standards it helped create. Webian is an operating system dedicated to running standards-based web apps.

The desktop version is currently built on the experimental Quantum Browser Runtime (qbrt) from Mozilla and the touch version is based on Android Things and GeckoView.

Webian Desktop 0.3
Webian Touch
Webian Touch running on a RasPad

The current approach I’m using for touch-based devices is to implement the system UI as an Android app which runs on top of Android Things. Android Things is basically a special version of Android which only runs a single app (no default system UI like a status bar or task manager is included), enabling device makers to have full control over the system UI.

Android Things has been used to build a range of commercial “smart displays” from Lenovo, LG and Samsung (though notably not Google’s own Home Hub smart display which is based on the more lightweight Google Cast platform). I’m using Android Things to build my own browser-based OS using Gecko via GeckoView with a native system UI in written in Java. The idea is that standard web apps can then be discovered and installed from the web using their Web App Manifest and Service Workers where available.

The upside of building on Android Things is that Google maintains the underlying OS and Android APIs, while I have full control over the system UI and can use Gecko via its existing well-supported Android widget layer. The downside is that it can probably never perform as well as B2G could have done on the same hardware because it requires that additional Java layer, the removal of which was one of B2G’s biggest benefits. And of course Android itself is controlled by Google.

Astian OS

Astian OS Logo

Astian OS calls itself a successor to Firefox OS, but it’s more of a spiritual fork than an actual derivative. It’s really a desktop Linux distribution with a love for web apps, based on Devuan (a fork of Debian).

Astian OS Launcher

IceWolf OS

IceWolf Logo

Originally known as Plánium OS, IceWolf OS (see what they did there?) is an open source project started in Brazil. According to its Twitter profile, IceWolf OS was inspired by Firefox OS and is built on web technologies.

After experiments with NodeJS & Chromium, Electron and NW.js, the project leader Patrick A Lima is now making an effort to move back to Gecko, along with its new Firefox OS inspired name. Unfortunately (as I have experienced myself), Gecko is much harder to embed due to the monolithic nature of its architecture.

Plánium OS

Mozilla Projects

My current job is working in Research & Development in the Emerging Technologies department of Mozilla. Many of the current projects in Emerging Technologies have their origins in Firefox OS.

The WebVR effort culminating in the Firefox Reality browser was first started as a potential new form factor for Firefox OS in VR headsets.

Firefox Reality

DeepSpeech was born as part of the Vaani voice assistant for Firefox OS, and may eventually be used in a voice browser called Firefox Listen.

Firefox Listen on Pocket

The current Mozilla IoT effort includes some early work on smart displays using Android Things and GeckoView.

Things Controller by Mozilla

Legacy

In the five years under Mozilla’s leadership around five million devices were shipped running Firefox OS. Since then over 50 million smart feature phones and millions of smart TVs have shipped running B2G code, and several new software projects have been derived from or inspired by it.

According to StatCounter, on mobile there are now more users of the KaiOS browser than Firefox (1.13% vs. 0.34%).

Mobile Browser Market Share, December 2018. Source: StatCounter

Their numbers show that KaiOS browser market share overtook Firefox on mobile some time in April 2018 and shows no signs of stopping.

Worldwide Mobile Browser Market Share 2017–2018. Source: StatCounter

With the current rate of growth of KaiOS and corresponding decline of Firefox market share, if KaiOS sticks with Gecko and Mozilla doesn’t manage to turn things around, it’s feasible that KaiOS could eventually become the biggest consumer of Gecko altogether.

But KaiOS is currently running a very old patched version of Gecko 48 and Panasonic’s My Home Screen 3.0 is running Gecko 34, while the latest version of Gecko in Firefox is 64.

KaiOS running Gecko 48
Panasonic My Home Screen 3.0 running Gecko 34

KaiOS Technologies and Panasonic are being prevented from upgrading to a newer version of Gecko because Mozilla removed all B2G code from the mozilla-central code repository, including the Gonk widget layer. Since then there have been significant architectural changes to Gecko as part of the Quantum project which make it very difficult for downstream projects to port B2G to the latest version.

This unfortunately means that the legacy of Firefox OS is currently tens of millions of phones and TVs across the world running an out of date browser engine which can probably never be updated. This potentially puts the users of those devices at risk, but also holds back the web platform as a whole.

The Future

So what will these companies do next? It’s very likely that without support from Mozilla, KaiOS will be forced to migrate from Gecko to Google’s Blink engine. In fact I understand there’s already work under way to do this, with support from the Google Chrome team.

The resulting architecture could look a lot like an Android/Chrome OS hybrid, something which has been speculated about for years. Panasonic could then use that same codebase, or just move to another software stack altogether.

A port of B2G to Blink is a huge undertaking, but if it comes to fruition it would further contribute to the WebKit/Blink monoculture we’re increasingly seeing on the web platform today with Chrome, Safari, Opera, Samsung Internet, Edge and Brave.

With both Opera and Microsoft having given up on their own browser engines in favour of building Chromium-based browsers, Mozilla really have the last ground-up alternative implementation of the web platform. This is hugely important because without multiple interoperable implementations, there can be no web standards.

There are ongoing efforts by Mozilla to target new Android-based platforms with Gecko-based browser apps on FireTV, Amazon Echo Show and VR headsets, as well as specialised browsers aimed at emerging markets. But while the rate of decline of Firefox market share has been slowing in recent times, with competitors converging on the Chromium code base and overall Mozilla market share still shrinking, the future of Gecko and the diversity and health of the web platform is looking a little bleak.

Conclusions

Since Mozilla discontinued Firefox OS, B2G source code has found its way into a surprising number of commercial products and now forms the basis of the third largest and fastest growing mobile operating system in the world. However, those devices are currently running out of date versions of Gecko, use locked down packaged app stores, are developed behind closed doors, and are arguably not a great showcase of the open web in the way B2G was originally intended to be.

I don’t necessarily think that smart feature phones and smart TVs would have been a great success for Mozilla had they pursued those product directions themselves with Firefox OS. Nor are those form factors necessarily the best showcase of the capabilities of the open web. However, the unexpected continuing legacy of B2G does serve to demonstrate just how much value still existed in what has proven to be a very versatile and scalable platform.

From low end $7 phones with tiny 2.4″ screens to high end $17,000 TVs with 77″ 4K screens, B2G has continued to perform.

Smart feature phone to Smart TV. Sources: KaiOS Technologies & Panasonic

In my personal opinion, Mozilla should seriously consider working with KaiOS Technologies to re-introduce the Gonk widget layer to mozilla-central, and re-claim some ownership over the B2G codebase. KaiOS could then use a slimmed down B2G architecture with local system services as had been the plan with the B2G Transition Project, and adopt modern web standards for installable web apps.

This would enable KaiOS to continue running an up-to-date version of Gecko on millions of devices rather than move to Blink, which many of their employees and partners might actually prefer, and could significantly contribute to the diversity and health of the web. I also believe this could open up enormous future opportunities for new product areas for Mozilla as well.

There are new form factors emerging today such as smart displays (Amazon Echo Show, Lenovo Smart Display, Google Home Hub) and tablet hybrid devices (Pixel Slate, ASUS Chromebook Flip) which could be a great fit for B2G and the open web.

Lenovo Smart Display. Source: Stacey on IoT
Pixel Slate. Source: Wired

With continuing explorations in voice, mixed reality and IoT there are many new potential product areas for Mozilla which could benefit from the B2G software stack.

Smart displays, for example, could be the ideal home for a voice-first trusted personal agent, consistent with Mozilla’s new three year strategy. Whilst early smart display products are being built on Google’s Android Things platform and that could be an option for Mozilla as well, even Google itself used Google Cast for its own Google Home Hub product, a more lightweight technology stack much more similar to B2G. B2G could give Mozilla and partners an edge in this market, with a platform less controlled by Google.

There could be other use cases for B2G in VR headsets, the smart home and countless connected devices we have yet to dream up.

Personally I would love to see Mozilla emerge from the comfort zone I fear it has retreated into in recent years, shipping Firefox browsers on others’ platforms where the web is always a second class citizen. I’d like to see some new big bets in products and partnerships where the web platform is allowed to take centre stage once again, where Mozilla isn’t constrained by the rules of other platforms.

The Web is the Platform, MWC 2012. Source: Madhava Enros

The web is the platform. Let’s allow it to shine.

Building the Web of Things

Originally posted on Mozilla Hacks.

Mozilla is working to create a Web of Things framework of software and services that can bridge the communication gap between connected devices. By providing these devices with web URLs and a standardized data model and API, we are moving towards a more decentralized Internet of Things that is safe, open and interoperable.

The Internet and the World Wide Web are built on open standards which are decentralized by design, with anyone free to implement those standards and connect to the network without the need for a central point of control. This has resulted in the explosive growth of hundreds of millions of personal computers and billions of smartphones which can all talk to each other over a single global network.

As technology advances from personal computers and smartphones to a world where everything around us is connected to the Internet, new types of devices in our homes, cities, cars, clothes and even our bodies are going online every day.

The Internet of Things

The “Internet of Things” (IoT) is a term to describe how physical objects are being connected to the Internet so that they can be discovered, monitored, controlled or interacted with. Like any advancement in technology these innovations bring with them enormous new opportunities, but also new risks.

At Mozilla our mission is “to ensure the Internet is a global public resource, open and accessible to all. An Internet that truly puts people first, where individuals can shape their own experience and are empowered, safe and independent.”

This mission has never been more important than in a time when everything around us is being connected to the Internet. As new types of devices come online, they bring with them significant new challenges around security, privacy and interoperability.

Many of the new devices being connected to the Internet are insecure, do not receive software updates to fix vulnerabilities and raise new privacy questions around the collection, storage and use of large quantities of extremely personal data.

Additionally, most IoT devices today use proprietary vertical technology stacks which are built around a central point of control and which don’t always talk to each other. When they do talk to each other it requires per-vendor integrations to connect those systems together. There are efforts to create standards, but the landscape is extremely complex and there’s still not yet a single dominant model or market leader.

The Web of Things

Using the Internet of Things today is a lot like sharing information on the Internet before the World Wide Web existed. There were competing hypertext systems and proprietary GUIs, but the Internet lacked a unifying application layer protocol for sharing and linking information.

The “Web of Things” (WoT) is an effort to take the lessons learned from the World Wide Web and apply them to IoT. It’s about creating a decentralized Internet of Things by giving Things URLs on the web to make them linkable and discoverable, and defining a standard data model and APIs to make them interoperable.

The Web of Things is not just another vertical IoT technology stack to compete with existing platforms. It is intended as a unifying horizontal application layer to bridge together multiple underlying IoT protocols.

Rather than start from scratch, the Web of Things is built on existing, proven web standards like REST, HTTP, JSON, WebSockets and TLS. The Web of Things will also require new web standards. In particular, we think there is a need for a Web Thing Description format to describe things, a REST style Web Thing API to interact with them, and possibly a new generation of HTTP better optimised for IoT use cases and use by resource constrained devices.

The Web of Things is not just a Mozilla Initiative, there is already a well established Web of Things community and related standardization efforts at the IETF, W3C, OCF and OGC. Mozilla plans to be a participant in this community to help define new web standards and promote best practices around privacy, security and interoperability.

From this existing work three key integration patterns have emerged for connecting things to the web, defined by the point at which a Web of Things API is exposed to the Internet.

Direct Integration Pattern

The simplest pattern is the direct integration pattern where a device exposes a Web of Things API directly to the Internet. This is useful for relatively high powered devices which can support TCP/IP and HTTP and can be directly connected to the Internet (e.g. a WiFi camera). This pattern can be tricky for devices on a home network which may need to use NAT or TCP tunneling in order to traverse a firewall. It also more directly exposes the device to security threats from the Internet.

Gateway Integration Pattern

The gateway integration pattern is useful for resource-constrained devices which can’t run an HTTP server themselves and so use a gateway to bridge them to the web. This pattern is particularly useful for devices which have limited power or which use PAN network technologies like Bluetooth or ZigBee that don’t directly connect to the Internet (e.g. a battery powered door sensor). A gateway can also be used to bridge all kinds of existing IoT devices to the web.

Cloud Integration Pattern

In the cloud integration pattern the Web of Things API is exposed by a cloud server which acts as a gateway remotely and the device uses some other protocol to communicate with the server on the back end. This pattern is particularly useful for a large number of devices over a wide geographic area which need to be centrally co-ordinated (e.g. air pollution sensors).

Project Things by Mozilla

In the Emerging Technologies team at Mozilla we’re working on an experimental framework of software and services to help developers connect “things” to the web in a safe, secure and interoperable way.

Project Things will initially focus on developing three components:

  • Things Gateway — An open source implementation of a Web of Things gateway which helps bridge existing IoT devices to the web
  • Things Cloud — A collection of Mozilla-hosted cloud services to help manage a large number of IoT devices over a wide geographic area
  • Things Framework — Reusable software components to help create IoT devices which directly connect to the Web of Things

Things Gateway

Today we’re announcing the availability of a prototype of the first component of this system, the Things Gateway. We’ve made available a software image you can use to build your own Web of Things gateway using a Raspberry Pi.

So far this early prototype has the following features:

  • Easily discover the gateway on your local network
  • Choose a web address which connects your home to the Internet via a secure TLS tunnel requiring zero configuration on your home network
  • Create a username and password to authorize access to your gateway
  • Discover and connect commercially available ZigBee and Z-Wave smart plugs to the gateway
  • Turn those smart plugs on and off from a web app hosted on the gateway itself

/media/59542bdd7abe70aba71d213ff814ba64

We’re releasing this prototype very early on in its development so that hackers and makers can get their hands on the source code to build their own Web of Things gateway and contribute to the project from an early stage.

This initial prototype is implemented in JavaScript with a NodeJS web server, but we are exploring an adapter add-on system to allow developers to build their own Web of Things adapters using other programming languages like Rust in the future.

Web Thing API

Our goal in building this IoT framework is to lead by example in creating a Web of Things implementation which embodies Mozilla’s values and helps drive IoT standards around security, privacy and interoperability. The intention is not just to create a Mozilla IoT platform but an open source implementation of a Web of Things API which anyone is free to implement themselves using the programming language and operating system of their choice.

To this end, we have started working on a draft Web Thing API specification to eventually propose for standardization. This includes a simple but extensible Web Thing Description format with a default JSON encoding, and a REST + WebSockets Web Thing API. We hope this pragmatic approach will appeal to web developers and help turn them into WoT developers who can help realize our vision of a decentralized Internet of Things.

We encourage developers to experiment with using this draft API in real life use cases and provide feedback on how well it works so that we can improve it.

Get Involved

There are many ways you can contribute to this effort, some of which are:

  • Build a Web Thing — build your own IoT device which uses the Web Thing API
  • Create an adapter — Create an adapter to bridge an existing IoT protocol or device to the web
  • Hack on Project Things — Help us develop Mozilla’s Web of Things implementation

You can find out more at iot.mozilla.org and all of our source code is on GitHub. You can find us in #iot on irc.mozilla.org or on our public mailing list.

The Story of Firefox OS

Originally posted on Medium.

The Story of Firefox OS

Mobile World Congress, 2013

I remember at a team dinner once Mitchell Baker, Mozilla’s Chairwoman and “Chief Lizard Wrangler”, talked about the importance of storytelling. She talked about how telling stories in the open source software community helps us to reflect on shared experiences and learn from them.

Well, I’m Ben and I’m a Mozillian. I’m a Software Engineer who worked on the “Boot to Gecko” project full time for five years and I have a story to tell.

As an engineer on the project I don’t quite have the full picture when it comes to the high level decision making, financials and business partnerships. But I was involved in the project for a long period of time, longer in fact than any of its co-founders, and gained quite a lot of insight at the engineering, design and product levels.

So I’d like to tell you my version of the story of Firefox OS, from the birth of the Boot to Gecko open source software project as a mailing list post and an empty GitHub repository in 2011, through its commercial launch as the Firefox OS mobile operating system, right up until the “transition” of millions of lines of code to the community in 2016.

During this five year journey hundreds of members of the wider Mozilla community came together with a shared vision to disrupt the app ecosystem with the power of the open web. I’d like to reflect on our successes, our failures and the lessons we can learn from the experience of taking an open source browser based mobile operating system to market.

This is not a short version of the story, in fact it’s over 7,500 words long, so below is a summary of the key points in case you don’t make it to the end! Hopefully it goes without saying that these are just my personal opinions.

Summary

Biggest Achievements

  • Design — Designed a simple and fresh looking mobile OS with some innovative features like edge gestures for switching windows. Came up with some great designs for how a browser based OS could work, even if the vision was never fully realised.
  • Engineering — Built a mobile operating system from scratch, entirely in the open, and got it to market in less than two years. As far as I know no other team has ever done this. Got web content rendering very close to bare metal on mobile devices with a very lean technology stack, in many cases beating the performance of native code on equivalent hardware. Pushed the envelope of the web with 30 experimental new web APIs and helped create new web app trends.
  • Product — Successfully launched 20 devices in over 30 countries, mobilised the Mozilla community to help run launch parties and created slick branding and marketing campaigns. Sold the message that “the web is the platform”, even if we failed to live up to it.
  • Partnerships — Won unprecedented support from the mobile industry. Repeatedly stole the show at MWC, the world’s biggest mobile event, and got big telecommunications corporations on board with Mozilla’s mission, with competing companies working together towards common goals.

Biggest Mistakes

  • Design — Started out by imitating what already existed, not something simpler, unique and focused on the web. Assumed we could change direction later, which turned out to be harder than we thought.
  • Engineering — In a rush to get to market we imitated the app store model with packaged apps, adding to the problem we set out to solve. Lost focus on what makes the web the web (URLs). Too much focus on the client side rather than the server side of the web stack.
  • Product — After realising “open” on its own doesn’t sell, ultimately chose a strategy to compete mainly on price, which is just a race to the bottom. Suffered from a lack of product leadership and direction after the initial launch.
  • Partnerships — Treated carriers and OEMs as our customers with a tick box approach to feature prioritisation, lost sight of the needs of end users and the original mission. Failed to secure a license to distribute updates directly to users.

What I Would Do Differently Today

  • Design — Start out with a very simple browser based design with a focus on web content rather than try to recreate every feature that other operating systems have. Be ruthless about leaving features out and not even attempt to play catch up.
  • Engineering — Have a clear separation between “chrome” and web content rather than try to force the web to do things it isn’t suited to. Create device APIs using REST & WebSockets on the server side of the web stack rather than privileged JavaScript DOM APIs on the client side. Create a community curated directory of web apps on the web rather than an app store of submitted packaged apps.
  • Product — Not try to create the cheapest smartphone. Create a mid-range tablet (and later possibly a smart TV stick) with a focus on web content, gaming and entertainment. Carve out a niche in developed markets first, rather than try to tackle emerging markets at scale.
  • Partnerships — Be a lot more forceful with partners about Mozilla leading the direction of the product roadmap and be willing to turn down opportunities if they don’t fit with our strategy. Provide software updates directly to end users and walk away from any chipset vendor who blocked us from distributing binaries of our own OS (this would have been very hard back then, but easier now).

Booting to the Web

The “Boot to Gecko” (B2G) open source project was announced in July 2011 with the goal of building “a complete, standalone operating system for the open web.”

“Mozilla believes that the web can displace proprietary, single-vendor stacks for application development. To make open web technologies a better basis for future applications on mobile and desktop alike, we need to keep pushing the envelope of the web to include, and in places exceed, the capabilities of the competing stacks in question.”

The key areas of work were identified as:

New web APIs: build prototype APIs for exposing device and OS capabilities to content (Telephony, SMS, Camera, USB, Bluetooth, NFC, etc.)

Privilege model: making sure that these new capabilities are safely exposed to pages and applications

Booting: prototype a low-level substrate for an Android-compatible device

Applications: choose and port or build apps to prove out and prioritize the power of the system

The first (unofficial) B2G project logo, by Sean Martell

There was no mention of commercial smartphones or mobile market share at this stage, merely the desire to “push the envelope of the web” to prove what it was capable of.

The Co-founders

The four co-founders of the project were Andreas Gal, Chris Jones, Brendan Eich and Mike Shaver.

B2G project co-founders (From left to right: Andreas, Chris, Brendan, Mike)

Andreas is a super smart engineer who wrote his PhD thesis on just-in-time compilation of high level programming languages and joined Mozilla in 2008 to build TraceMonkey, the first JavaScript just-in-time compiler. He’d recently finished writing PDF.js, a PDF renderer written in JavaScript, and wanted to show what else JavaScript was capable of.

Chris got his PhD in Computer Science from Berkeley, joined Mozilla in 2009 and had worked on PDF.js with Andreas.

Brendan invented JavaScript and co-founded Mozilla in 1998. When B2G was announced he was Mozilla’s CTO, and the executive sponsor of the project.

Mike was a founding member of Mozilla in 1998 and had served as the VP of Engineering and VP of Technical Strategy, but left Mozilla shortly after the B2G project was announced.

I applied to work on the B2G project at Mozilla the day after it was announced so I wasn’t employed there at the time of the announcement, but my understanding is that it ruffled some feathers. Partly because it was announced on a public mailing list before any significant internal discussion, and partly because most people (including senior executives) didn’t know much about the project and weren’t expecting the announcement.

It had the feel of a skunkworks project and was run a lot like a startup, partially outside the normal operation of the Mozilla Corporation.

The Building Blocks

Starting from an empty GitHub repository the co-founders built a small initial team to work on “Gonk” (a “low-level substrate” based on the Linux kernel and Android Hardware Abstraction Layer), “Gecko” (an app runtime and device APIs built on the web engine of Firefox) and “Gaia” (a smartphone user interface built with web technologies HTML, CSS and JavaScript).

B2G Architecture

Some code was re-used from the Android Open Source Project but what was unique about this architecture was that it cut out the entire Java layer of Android, with Gecko rendering web content as close to bare metal as the team could get it. This would give B2G a real edge when it came to performance and would help put web content on a level footing with native code.

This distributed and small but growing team had their first in-person work week in Taipei in December 2011 where Mozilla’s Taiwan office was in the process of being fitted out, in a building overlooking the iconic Taipei 101. It was at this work week at the Grand Hyatt Hotel that the first ever phone call was made from a web page without the use of a browser plugin. The basics were starting to come together.

Taipei, December 2011

The Prototype

At a follow-up work week in Paris in February 2012, the team (now including Michael Vines from Qualcomm and our one-man UX team Josh Carpenter) raced up until the last minute to put together a prototype to showcase at Mobile World Congress in Barcelona that same month.

Paris, February 2012

We succeeded in delivering the initial components described in the original B2G announcement far enough to have a proof of concept demo running on existing Android hardware. We went to MWC with a prototype running on a Samsung Galaxy II smartphone. This was revealed in a joint announcement with our first partner, Telefónica, with additional support pledged from Adobe, Qualcomm and Deutsche Telekom. At the press conference Telefónica demoed B2G software running on Qualcomm reference hardware.

B2G prototype, MWC 2012, Barcelona

This caught the eye of other mobile carriers who were on the lookout for a platform with the potential to challenge Android, to create more choice in the mobile operating system space and wrangle some control back from the dominant Google and Apple.

As a non-profit with a mission to “promote openness, innovation & opportunity on the web” Mozilla was selling a unique vision — not that our own app platform would somehow become the “third platform” on mobile, but that the open web could fulfil that role. Like on desktop, the ubiquity and scale of the web could make it the only viable contender to the incumbent app platforms, with Mozilla leading the way.

Partnerships

With the support of new partners, Mozilla promised to return the following year with commercial devices ready to go to market.

One of the first partners was Telefónica, who had been working on their own “Open Web Device” project. At first the idea was that Mozilla could provide the underlying Gecko-based platform and Telefónica would add their own Open Web Device front end and contribute their expertise in certifying and taking devices to market.

Working with a huge telecommunications corporation like Telefónica was new ground for open source shop Mozilla and I’d say initially there was a bit of a culture clash. There were a few points of tension and for a while Mozilla and Telefónica worked on their own competing front ends.

Telefónica and Mozilla alternative B2G front ends

It quickly became clear that it made more sense to pool our resources and work on a single mobile front end, and soon Mozilla and Telefónica engineers were working side by side. Both teams of engineers really embraced this approach and it soon felt like we were all one team.

Deutsche Telekom was another partner who made big early contributions and with mobile carriers on board it became easier to get OEMs interested. The first two OEMs were Chinese manufacturing competitors ZTE and TCL.

One of the reasons these partnerships were so important was that Qualcomm, the first chipset manufacturer we were working with, would not license their firmware and device drivers to us directly, it had to be through a carrier or OEM. This meant that we legally couldn’t ship complete builds or updates of the OS directly to users, which due to the economics of mobile OS updates would turn out to be a serious problem for us.

Firefox OS 1.0

For the next year Mozilla worked intensively with our partners to build a full production quality mobile operating system.

Design

The brief given to the UX team was basically not to do anything too surprising for the first iteration. Stick to a grid of app icons like Android and iOS and keep it simple, we’d figure out how to differentiate the user experience later on. First we would prove it was possible to build the kind of UI that already existed using web technologies.

The designs the team came up with were pretty slick, but in retrospect starting out by imitating what already existed on the assumption we could change direction later turned out to be a mistake.

Implementation

On the Engineering side we would prove that it was possible to build a smartphone using open web technologies, with the whole user interface written in HTML, CSS and JavaScript.

It was a requirement for all of the built-in Gaia apps to function offline when an Internet connection was patchy or unavailable, so that the user could continue to make phone calls, take photos and listen to music etc. At first we started to use AppCache for this purpose, which was the web’s first attempt at making web apps work offline. Unfortunately we soon ran into many of the common problems and limitations of that technology and found it didn’t fulfil all of our requirements.

We also needed to invent lots of new APIs to talk to the underlying hardware of a smartphone from JavaScript, and a privilege model which made that possible. The existing security model of the web allows browsers to safely execute untrusted code by restricting it to a very limited sandbox. In order to provide web content access to much more privileged functions like making phone calls and sending text messages it was thought a new security system was needed.

In order to ship version 1.0 of B2G on time, it was decided that the simplest way to achieve both the offline and security requirements was to introduce the concept of packaged apps to Gecko. A package of resources could be “installed” onto the device to work offline and cryptographically signed to verify that its code is trusted.

Packaged apps solved our immediate problems but they are not truly web apps because they don’t have real URLs on the web and they ultimately have to be signed by a central authority (like Mozilla) to say that they’re safe. I argued against the packaged app approach at the time on the basis that it wasn’t really the web, but nobody could come up with a more webby solution we thought we could implement and ship on time.

At a work week in Telefónica’s offices in Barcelona in July 2012 it was decided to go ahead with packaged apps as a stop-gap solution until we came up with something better. This was another decision which I think turned out to be a mistake because, as well as creating significant technical debt, it set us on a path which would be difficult to turn back from.

Race to the Finish

Anyway, all of this effort culminated with a buzzing work week in Berlin in January 2013 where Mozilla worked side by side with partners, including chipset maker Qualcomm, carrier competitors Telefónica and Deutsche Telecom and manufacturing competitors ZTE and TCL, with the common goal of shipping version 1.0.

This event, hosted by Deutsche Telekom, turned out to be one of the most memorable moments in B2G’s development. Everyone worked together in the same room with a super fast Internet connection, and a friendly competition had been set up between the front end and back end teams to see who could fix all of their bugs first. The results were displayed in real time on a projector so everyone could monitor our progress. There was incredible energy in the room for the whole week while over 200 bugs were fixed.

Berlin work week, January 2013

By the end of the week Andreas Gal declared “zarro boogs” and Firefox OS version 1.0 was born. (The front end team won, by the way!)

Mobile World Congress 2013

The first commercial version of Firefox OS to ship was actually 1.0.1, once all of the necessary testing and certification had been completed. The first commercial devices were announced at Mobile World Congress in Barcelona (the biggest mobile conference in the world) in February 2013. They were the ZTE Open and the Alcatel One Touch and they would launch via Telefónica and Deutsche Telekom in Brazil, Colombia, Hungary, Mexico, Montenegro, Poland, Serbia, Spain and Venezuela. LG and Huawei had also committed to shipping devices.

ZTE Open & Alcatel One Touch smartphones

The announcements were made at a press conference the day before MWC. We were expecting around 250 people, but on the day over 700 press, media and analysts showed up.

Firefox OS press conference, MWC 2013, Barcelona (source)

We managed to get an unprecedented 18 CEOs of telco companies up on stage to express their support for the project, which I understand took even the GSMA by surprise.

CEOs on stage at press conference, MWC 2013, Barcelona (source)

I think it’s fair to say Firefox OS was really the star of the show that year. We made the front page of the conference magazine every day of the event, we got around 80,000 press results on Google News, and Firefox OS was trending on Twitter with the Oscars! It was a PR dream.

MWC magazine, MWC 2013, Barcelona (source)

We had a very cool custom designed booth at the conference (in addition to a presence on the Telefónica and Deutsche Telekom stands), with a reception desk, a giant fox tail overhead and demo stations dotted around.

Firefox OS booth, MWC 2013, Barcelona (source)

I was helping to give demos and press interviews at the event and I can tell you that the footfall was non-stop. There was a huge amount of interest from carriers, OEMs and press in what we were doing.

An Android mascot, trying to get in on the attention!

At an event which is infamous for its “booth babes” (companies hiring professional models to show off their wares), something which I think really made the Firefox OS booth stand out was that most of the people manning our stand had actually worked on the project, and many of them were even volunteers. This was something that really surprised some of the conference attendees, that people would be passionate enough about the project to give up their time freely to help promote it.

Mobile World Congress, Barcelona, February 2013 (source)

This volunteer community involvement continued long after MWC when the devices actually started rolling out in the first launch countries. Launch events in Spain, Poland, Venezuela, Colombia and others were manned by local members of the Mozilla community.

There’s a great video made in 2014 which tells the story up to the 1.0 launch, embedded below.

Firefox OS The Journey, the story of the v1 launch

Firefox OS 1.x

Firefox OS had made it to market. But before the 1.0 launch parties had even got going, we were already back at the office hard at work figuring out what came next.

At first the focus was mainly on adding basic smartphone features like MMS which had been missing from the 1.0 release, then the focus shifted to better development tools for app developers and an awful lot of requirements from partners.

We quite quickly slipped into a pattern where we were treating the mobile carriers and OEMs as our customers. They had a never-ending list of requirements which were essentially copy-pasted from their Android device portfolios. They wanted Firefox OS to reach feature parity with Android.

I remember spending an entire week shut in a room in Qualcomm’s offices in San Diego in 2013 with a group of very hard-working South Koreans from LG, making sure that all the built-in Firefox OS apps started quicker than equivalent Android apps on the same hardware. For confidentiality reasons they wouldn’t tell us how how many milliseconds the Android apps actually took to launch on their device, only whether or not we were quicker. We’d just have to keep hacking away until we had the faster launch time. The LG Fireweb finally launched in October 2013 running Firefox OS 1.1.

LG Fireweb with Firefox OS 1.1, October 2013

Over the course of 2013 and 2014 versions 1.1, 1.2, 1.3 and 1.4 were released. Eventually it became clear that we were just chasing the coat tails of Android, and with Android having a five year head start on us we had no chance of catching them. If we wanted Firefox OS to actually compete in the market and gain any significant market share, we had to differentiate.

Differentiation

Everyone basically agreed that we couldn’t compete with the likes of Android and iOS on their own terms. We couldn’t catch up with Google on Android features and we could never out-Apple Apple on design. Mozilla was used to punching above its weight and had taken on titans before and won, but we wouldn’t win if we played by their rules – we had to play by our own rules.

The way I remember it is that there were basically two schools of thought about how to differentiate Firefox OS.

The Web is the Platform

One school of thought was that we should not try to reach feature parity with our competitors but instead look for a different take on the user interface. We should come up with a much simpler UI which worked to the strengths of the web.

A big challenge we had was that the press would often compare Firefox OS with its competitors based on how many apps were in the app store. Firefox OS was never meant to have a single central app store, the idea was that the web was the platform. Unfortunately some of the early technical decisions we made around packaged apps had backed us into a corner where we were essentially dependent on the few thousand apps submitted to the Mozilla-run Firefox Marketplace.

We tried to patch over the problem by partnering with a startup called Everything.me who maintained a directory of web apps, in order to boost our content catalogue. But ultimately by relying on a central directory we’d ended up just adding to the centralised app store problem we’d set out to solve.

We talked about refocusing on real web content (by far the biggest collection of digital content on the planet) and coming up with an innovative user interface optimised for the interlinked nature of the web and the “flow” of navigating from page to page via hyperlinks. Working to the strengths of the open web would allow us to come up with something genuinely different to the competition.

Connecting the Next Billion

The other school of thought was based on the assumption that what we really needed was scale, and to get scale our best bet was to target emerging markets. They were the real growth areas in the mobile industry.

We did some market research and user studies in developing countries and it quickly became clear that much of the world’s population was simply priced out of the smartphone market. What was needed was an affordable smartphone that people in places like India and Bangladesh could actually buy.

If we could crack emerging markets then we could very quickly scale Firefox OS. Then at least we could escape the “other” category on those market share pie charts and start to gain some traction.

The $25 Smartphone

In mid-2013 Li Gong took the helm of Firefox OS. He was responsible for starting and running Mozilla’s China and Taiwan operations, but took on the additional role of “Senior VP of Mobile Devices”.

Li Gong, MWC 2014, Barcelona (source: CNET)

Li was firmly in the “affordable smartphone” camp and championed the concept of a $25 smartphone, which was felt was the price point we needed to hit if we were to get scale in emerging markets.

The team pulled off an incredible feat of engineering by managing to get Firefox OS to run on some very low end hardware based on a Spreadtrum chipset with only 128MB RAM — way below the minimum requirements for Android at the time.

We went to MWC 2014 with the “$25 smartphone”, new partnerships with low-cost handset makers Intex and Spice and a new message around “connecting the next billion”, which seemed a good fit with Mozilla’s mission. This message resonated well at MWC, with the mobile industry wondering where to grow next.

Proof of concept $25 smartphone, MWC 2014, Barcelona (source)

With our new affordable smartphones Firefox OS started to expand into Asia (India, Bangladesh), Latin America (El Salvador, Panama, Nicaragua, Guatamala) and Eastern Europe (Czech Republic).

Firefox OS bus, India

Unfortunately the execution was not so great. Firstly, the $25 smartphone eventually ended up costing more like $33 by the time it got to market. Also whilst Firefox OS “ran” on 128MB RAM you couldn’t really multi-task in the way you’d usually expect on a smartphone. Another serious problem was the lack of a key app, Whatsapp, which was essential for many of these markets. We failed to convince WhatsApp to make a web version, or even let us write one for them. These shortcomings, coupled with some hardware quality issues (particularly around the camera), meant that these affordable smartphones didn’t do very well in the market.

The problem with competing on price is that you soon get into a race to the bottom and whoever has the biggest economy of scale ends up winning. Google were quick to notice this chink in their armour and responded with Android One, affordable devices targeted at emerging markets. Android One arguably didn’t really do very well either, but it was enough to expose the fact that simply competing on price was not going to be a winning strategy.

Just because people in emerging markets may not have much disposable income, doesn’t mean they’ll settle for poor quality. They don’t want a cheap knock off of an iPhone, they want an iPhone. If they can’t have an iPhone they’d rather stick with their feature phone. It’s affordable, it works and it has a far superior battery life.

Firefox OS 2.0

Most of the work to create the $25 smartphone had been done by engineers in the Taipei office with a special release called 1.3t. Meanwhile, other teams had been thinking about what a version 2.0 might look like.

In April 2013 the UX team held a summit in London where they got together to discuss future directions for the user experience of Firefox OS. I was privileged to be invited to participate. We brainstormed about what was unique about the experience of the web and how we might create a unique user experience which played to those strengths.

UX Summit, London, April 2013

A big focus was on “flow”, the way that we can meander through the web by following hyperlinks. The web isn’t a world of monolithic apps with clear boundaries between them, it is an experience of surfing from one web page to another, flowing through content.

Haida

In the coming weeks the UX team would create some early designs for a concept (eventually code-named “Haida”) which would blur the lines between web apps and websites and create a unique user experience which would flow like the web does.

This would eventually include not only the universal “Rocketbar”, which would be accessible across the whole OS and seamlessly adapt to different types of web content, but also “sheets” which would split single page web apps into multiple pages which you could swipe through with intuitive edge gestures. It would also eventually include a content model based around live apps which you could surf to, use, and then bookmark if you chose to, rather than monolithic apps which you have to download and install from a central app store before you can use them.

In June 2013 a small group of designers and engineers met in Paris to develop a throwaway prototype of Haida, to rapidly iterate on some of the more radical concepts and put them through user testing.

Haida Team Meetup, Paris, June 2013

Below is a video Josh Carpenter created in October 2013 to communicate the Haida design concept./media/99095ebaf6616bbc1fbeb92d4430328a

This drastic change was too big to get out in one release, so Haida was split into different phases. Version 2.0 brought a visual refresh and the implementation of edge gestures for moving between windows. Version 2.1 included the integration of the browser into the main system UI to start to create the unique browser-based experience.

Firefox OS 2.x

But by version 2.2 the focus on the Haida initiative started to slip. Competing feature requests started coming in from partners to create greater feature parity with Android, improve performance and add support for right-to-left languages so that partners could ship devices in more countries.

The full Haida vision was never fully realised and people started talking about another change of direction for a version 3.0.

Firefox OS 3.0

After better than expected sales results reported by partners for the first device launches, the $25 smartphone initiative had been a disappointment. With no clear point of differentiation having yet fully materialised to continue the momentum of the initial launches, partners’ sales numbers were starting to look disappointing. A loss of confidence started to emerge and disagreements about which direction to take the product ultimately resulted in the departure of several key executives.

Li Gong left Mozilla to create a fork of Firefox OS through his startup Acadine Technologies, with $100m of funding from several Chinese government-backed organisations. When Li left he took a lot of engineers from Mozilla’s Taipei office with him, which caused a set back for Mozilla and some bad blood. This turned out to be a real waste because the $100m eventually failed to materialise and Acadine was wound down.

Andreas Gal also left to create Silk Labs, taking Chris Jones and other key members of the original B2G team with him. With Chris Lee, the head of Product, having also made a departure there was a real vacuum of leadership at the top of the Firefox OS team.

This was around the time that Chris Beard took over as CEO of Mozilla and it fell to him to temporarily take charge of the Firefox OS organisation while they tried to hire someone. Chris is a great CEO and as a long-time Mozillian has a deep intuitive sense of what Mozilla is, but as CEO he didn’t really have the time or capacity to actively lead Firefox OS from a product point of view.

3.0 Ideation Process

With a lack of any real product direction, the “v3 Ideation Process” was kicked off at a last minute whole company meetup in Portland in December 2014. The idea was to crowd-source ideas for a new direction for Firefox OS, from individual members of the Firefox OS team and the wider community.

v3 ideation process, Portland, December 2014

On the surface, for people like me who had well developed ideas about what direction I thought we should take, this seemed like a great opportunity. But as around 36 ad-hoc groups were set up all brainstorming different ideas, good ideas started to get lost in the noise. With no real constraints put on the ideation process and an insufficient process for evaluating them, people were coming up with all sorts of suggestions from smart watches to reinventing the concept of currency!

In the end no clear direction emerged and the 3.0 release was downgraded to a “2.5” release with some hurriedly put together features.

The Pivot

With an underwhelming 2.5 release and still suffering from a complete lack of product leadership, during 2015 Firefox OS started to stall.

Eventually, Ari Jaaksi was hired as the new head of “Connected Devices”. The Firefox OS team was renamed the Connected Devices team and at a whole company meetup in Orlando the “pivot” was announced.

Ari Jaaksi, Orlando All Hands, December 2015

Mozilla would stop our work to build and ship smartphones through carrier partners and we would instead start to prototype user experiences around connected devices and the Internet of Things. The technology created for Firefox OS would be used as a starting point and Firefox OS could live on with devices like Smart TVs (which had been showing some promise through a partnership with Panasonic), but we would no longer focus on smartphones.

At the end of Ari’s talk a motivating video was played on the big screen (embedded below)./media/40756ab8866005cb60c8093001f6e17eFirefox OS — The Next Chapter, played at Orlando All Hands December 2015

Nobody noticed. They were too busy chatting among themselves and trying to absorb this news and what it would mean.

The premise of Ari’s talk was that Firefox OS had set out to compete with Android and iOS and it had failed. Firefox OS was too late to market, the app store hadn’t taken off and the smartphone war had been won. It was time to move onto the next big thing — the Internet of Things.

This analysis was a little frustrating to me because I’d never felt that what we’d set out to do was to make Firefox OS the third mobile app platform, it was about pushing the envelope of the open web to make it a competitive platform for app development. It was true that the project was stalling, but we’d had some really good ideas with Haida, what we’d been lacking was focus.

However, I tried to embrace this new direction as I’d long suspected that tablets would have been a better form factor for Firefox OS than smartphones in the first place, and the Smart TV work was showing promise. These were both “connected devices” after all.

Some of the Connected Devices products I proposed

The “Product Innovation Process” was kicked off, another bottom-up ideation process to generate product ideas, but this time with much more structure (arguably too much). Connected Devices operated much like an internal startup incubator, teams were sent on “lean startup” training and we all had to become experts in the emerging field of the Internet of Things as we pitched product ideas to a “Product Innovation Board”.

The Transition

The announcement in Orlando was quickly jumped upon by some inside Mozilla as a signal that “Firefox OS is dead”. With all the resources that had been put into Firefox OS over the years, other teams at Mozilla had been feeling the pinch. The flagship Firefox team and supporting platform team had been complaining about a lack of resources for a while, and with Firefox market share slipping the finger of blame was pointed at Firefox OS.

I’d say some of this resentment was maybe justified but it started to spiral out of control and Firefox OS soon became a scapegoat for everything that wasn’t going well at Mozilla. There was a general feeling that Mozilla had “bet the farm” on Firefox OS and it hadn’t paid off. Significant political pressure started to grow inside Mozilla to remove all traces of B2G from the codebase and re-assign resources to our flagship product, Firefox.

The Connected Devices team was downsized, with around 40 people being made redundant. Meetings were held to evaluate what value the existing B2G codebase still had with our new direction, and how much of it could be removed. The argument was the B2G code was slowing down Firefox development and the cost of maintaining it couldn’t be justified.

With some Connected Devices projects (like the tablet and smart TV projects) still planning to use parts of the B2G code, we agreed on a plan to drastically slim down the B2G architecture. We’d completely remove the mozApps packaged app runtime in favour of new web standards, and remove around 30 device APIs we’d added to Gecko which were clearly never going to become web standards, in favour of a new web services based approach.

B2G OS Architecture Transition

The “B2G Transition Project” was started and the Mozilla community were involved in the process so that any smartphone-specific code could be transferred to community ownership if anyone wanted to continue to maintain it. The community really rose to this challenge and got involved in the process, ironically generating the greatest level of community involvement the B2G project had ever seen.

Although we successfully managed to remove the mozApps runtime and 10 of the 30 device APIs from Gecko very quickly, internal political pressure continued to grow to completely remove all traces of B2G from the Mozilla codebase. The tablet and smart TV projects were stopped and the Connected Devices team started to focus their efforts elsewhere.

The module ownership system of the Mozilla project is technically entirely independent of the Mozilla Corporation’s reporting structure. As a last ditch effort the module owner of B2G (then Fabrice Desré) disputed the removal of the B2G module from the Mozilla project and the decision was escalated all the way to the top of the tree, but ultimately the decision was to remove it.

Technically the new community-owned “B2G OS” could still continue as a fork, but with infrastructure being shut down and no Mozilla staff being paid to maintain the code, a fork was never really going to be practical. At its peak around 300 full time Mozilla staff were working on the Firefox OS project and the community didn’t have anywhere near that number of skilled full time resources.

Boot to Gecko was effectively dead.

Conclusions

Measuring Success

I’m incredibly proud of what we achieved with Firefox OS. If you measure its success against the original stated goals I think the project went way beyond its initial objectives. We went beyond a technical prototype to successfully ship 20 different commercial devices in over 30 countries, including smartphones and smart TVs. Apart from the incredible feats of engineering, prototyping 30 new APIs for the web platform and building possibly the most complex JavaScript codebase ever created, we built a mobile operating system from scratch and got it to market in less than two years. As far as I know no other team has ever done this.

Just some of the 20 Firefox OS products that went to market

I don’t actually know how many individual Firefox OS devices were sold in the end because our partners didn’t share their sales numbers with us and we didn’t ask them. As an educated guess I’d say it was probably around 5 million (give or take a million or so). To me that’s an incredible achievement, it’s a lot more than other “alternative” mobile operating systems have achieved and certainly the most successful project I’ve ever been involved in. But in an industry which ships over a billion units a year that’s not even half a percent of market share!

For me it was never about Firefox OS being the third mobile platform. It was always about pushing the limits of web technologies to make the web a more competitive platform for app development. I think we certainly achieved that, and I would argue our work contributed considerably to the trends we now see around Progressive Web Apps. I still believe the web will win in the end.

At some point along the way, the team lost sight of our original goals and somehow defaulted to trying to become the third smartphone platform alongside Android and iOS, with market share as the only metric of success. The B2G team grew from four co-founders to a hundred people in a very short space of time and it’s not uncommon for a startup which grows very fast to lose its mission and focus along the way. (That’s what Firefox OS felt like to me, it was a startup inside Mozilla).

A Matter of Timing

A common narrative is that Firefox OS was simply too late to market. Talking with experts from Shenzhen in China I’m actually no longer sure that’s true. They’ve suggested to me that if Firefox OS started today, the licensing issues which caused us so many problems and forced us to go through mobile carriers rather than ship software updates directly to users would no longer be a problem. As mobile hardware becomes more commoditised it would be trivial today to find white label hardware on which to run our software. There’s also a real desire from OEMs to loosen their dependence on Google and differentiate themselves. It’s possible that rather than being five years too late, Firefox OS was actually five years too early!

Those of us who work on the web like to think that what we do is really cutting edge. But the reality of web standardisation is that it takes a long time. New features will always come to native code first before they make it through the standardisation process to a cross-platform solution. I’d argue that the web is really more suited to more mature hardware platforms which have become more of a commodity, and that eventually it will win out on mobile in the same way it has on desktop. It could be that the time is quickly approaching when a web-oriented mobile OS could be successful.

The Right Decision?

It’s been over a year now since the announcement was made in Orlando and I’ve had plenty of time to process the various stages of grief associated with the end of Firefox OS: denial, anger, bargaining, depression and acceptance!

Now that the dust has settled I can honestly say that on balance I still think cancelling Firefox OS completely was the wrong decision. The project had lost momentum due to a lack of product leadership over an extended period of time, but I believe the decision to kill it was ultimately based more on internal political reasons than sound business or technical ones.

As the mobile market has matured and become more commoditised, web technologies have also matured. The opportunity is still there, in fact the timing may now be better than ever. Although I understand the reasons the project was stopped, I feel Firefox OS was ended prematurely, before it had the chance to reach its full potential. What it needed was a reboot and some ruthless and clear direction, which unfortunately it didn’t get.

Many will argue that I’m delusional and that it’s easy for me to say this, after all I wasn’t the one who had to sign the eye-watering cheques that needed signing to keep a project of that scale running. They’ll say that we had our chance and we blew it. Mozilla should now be focusing all of its efforts on its flagship product, Firefox. For me, that’s a little too inward looking. In many ways the web has now moved beyond the browser, it’s time to look to the future and all of the new challenges which face the Internet.

Lessons Learned

Sadly, it would be politically impossible to reboot B2G inside Mozilla today, but if it was possible there are a number of things I would do differently a second time around.

Design — I’d start with a very simple browser-based user interface with a focus on web content, rather than try to replicate all the features of other mobile operating systems.

Engineering —Have a clear split between “chrome” and web content rather than try to force the web to do things it isn’t suited to. Create device APIs using REST & WebSockets on the server side of the web stack rather than privileged JavaScript DOM APIs on the client side (JavaScript is not what makes the web the web, URLs are). Create a community curated directory of web apps on the web rather than an app store of developer-submitted packaged apps.

Product — Not try to create the cheapest smartphone. Create a mid-range tablet (and later possibly a TV stick) with a focus on web content, gaming and entertainment. Carve out a niche in developed markets first, rather than try to tackle emerging markets at scale.

Partnerships — Be a lot more forceful with partners about Mozilla leading the direction of the product roadmap and be willing to turn down deals if they don’t fit with our strategy. Provide software updates directly to end users and walk away from any chipset vendor who blocked us from distributing binaries of our own OS (this would have been very hard back then, but easier now).

Footnote

So that’s my story. It’s pretty long so if you managed to get this far without skipping at least some of the boring bits then I’m impressed!

My main hope in documenting all of this is that Firefox OS is remembered as more than just a footnote in tech history which says “a failed mobile OS from Mozilla”.

A lot of super smart and awesome people put an awful lot of love and hard work into the Boot to Gecko project. It’s something I’m incredibly proud of, and they should be too. It might not have matched up to the expectations imposed on it from the outside world, but from my point of view Firefox OS was a success and certainly the highlight of my career so far! (I was actually an intern at Google in 2008 when they shipped the first Android smartphone, but for me the buzz of Firefox OS was far bigger than that).

Some of the big brains behind Firefox OS (and me), Whistler, June 2015

I still believe deeply in Mozilla’s mission and I also think there’s an exciting new chapter coming to Mozilla soon, with some new emerging technologies which I’m itching to tell you about. But that’s a story for another day.

For now, please forgive me the indulgence of ending with a video my wife and I created in 2014, with our dodgy Bob Dylan parody and a collection of photos which capture the incredible journey that was Firefox OS. (See if you can spot the cameo by Sir Tim Berners-Lee, trying out Firefox OS over lunch!)

The Times They Are a Changin’, Open Web Remix by Ben and Siobhan

[Edit 2017–03–06]:

  • Clarified Brendan Eich’s role at Mozilla
  • Mentioned Telefónica’s demo on Qualcomm reference hardware at MWC 2012
  • Noted the lack of WhatsApp being a major problem in many markets
  • Credited B2G logo designer

Pinned Apps – An App Model for the Web

(re-posted from a page I created on the Mozilla wiki on 17th December 2014)

Problem Statement

The per-OS app store model has resulted in a market where a small number of OS companies have a large amount of control, limiting choice for users and app developers. In order to get things done on mobile devices users are restricted to using apps from a single app store which have to be downloaded and installed on a compatible device in order to be useful.

Design Concept

Concept Overview

The idea of pinned apps is to turn the apps model on its head by making apps something you discover simply by searching and browsing the web. Web apps do not have to be installed in order to be useful, “pinning” is an optional step where the user can choose to split an app off from the rest of the web to persist it on their device and use it separately from the browser.

Pinned_apps_overview

”If you think of the current app store experience as consumers going to a grocery store to buy packaged goods off a shelf, the web is more like a hunter-gatherer exploring a forest and discovering new tools and supplies along their journey.”

App Discovery

A Web App Manifest linked from a web page says “I am part of a web app you can use separately from the browser”. Users can discover web apps simply by searching or browsing the web, and use them instantly without needing to install them first.

Pinned_apps_discovery

”App discovery could be less like shopping, and more like discovering a new piece of inventory while exploring a new level in a computer game.”

App Pinning

If the user finds a web app useful they can choose to split it off from the rest of the web to persist it on their device and use it separately from the browser. Pinned apps can provide a more app-like experience for that part of the web with no browser chrome and get their own icon on the homescreen.

Pinned_apps_pinning

”For the user pinning apps becomes like collecting pin badges for all their favourite apps, rather than cluttering their device with apps from an app store that they tried once but turned out not to be useful.”

Deep Linking

Once a pinned app is registered as managing its own part of the web (defined by URL scope), any time the user navigates to a URL within that scope, it will open in the app. This allows deep linking to a particular page inside an app and seamlessly linking from one app to another.

Pinned_apps_linking

”The browser is like a catch-all app for pages which don’t belong to a particular pinned app.”

Going Offline

Pinning an app could download its contents to the device to make it work offline, by registering a Service Worker for the app’s URL scope.

Pinned_apps_offline

”Pinned apps take pinned tabs to the next level by actually persisting an app on the device. An app pin is like an anchor point to tether a collection of web pages to a device.”

Multiple Pages

A web app is a collection of web pages dedicated to a particular task. You should be able to have multiple pages of the app open at the same time. Each app could be represented in the task manager as a collection of sheets, pinned together by the app.

Pinned_app_pages

”Exploding apps out into multiple sheets could really differentiate the Firefox OS user experience from all other mobile app platforms which are limited to one window per app.”

Travel Guide

Even in a world without app stores there would still be a need for a curated collection of content. The Marketplace could become less of a grocery store, and more of a crowdsourced travel guide for the web.

Pinned_apps_guide

”If a user discovers an app which isn’t yet included in the guide, they could be given the opportunity to submit it. The guide could be curated by the community with descriptions, ratings and tags.”

3 Questions

Pinnged_apps_pinned

What value (the importance, worth or usefulness of something) does your idea deliver?

The pinned apps concept makes web apps instantly useful by making “installation” optional. It frees users from being tied to a single app store and gives them more choice and control. It makes apps searchable and discoverable like the rest of the web and gives developers the freedom of where to host their apps and how to monetise them. It allows Mozilla to grow a catalogue of apps so large and diverse that no walled garden can compete, by leveraging its user base to discover the apps and its community to curate them.

What technological advantage will your idea deliver and why is this important?

Pinned apps would be implemented with emerging web standards like Web App Manifests and Service Workers which add new layers of functionality to the web to make it a compelling platform for mobile apps. Not just for Firefox OS, but for any user agent which implements the standards.

Why would someone invest time or pay money for this idea?

Users would benefit from a unique new web experience whilst also freeing themselves from vendor lock-in. App developers can reduce their development costs by creating one searchable and discoverable web app for multiple platforms. For Mozilla, pinned apps could leverage the unique properties of the web to differentiate Firefox OS in a way that is difficult for incumbents to follow.

UI Mockups

App Search

Pinned_apps_search

Pin App

Pin_app

Pin Page

Pin_page

Multiple Pages

Multiple_pages

App Directory

App_directory

Implementation

Web App Manifest

A manifest is linked from a web page with a link relation:

  <link rel=”manifest” href=”/manifest.json”>

A manifest can specify an app name, icon, display mode and orientation:

 {
   "name": "GMail"
   "icons": {...},
   "display": "standalone",
   "orientation": “portrait”,
   ...
 }

There is a proposal for a manifest to be able to specify an app scope:

 {
   ...
   "scope": "/"
   ...
 }

Service Worker

There is also a proposal to be able to reference a Service Worker from within the manifest:

 {
   ...
   service_worker: {
     src: "app.js",
     scope: "/"
   ...
 }

A Service Worker has an install method which can populate a cache with a web app’s resources when it is registered:

 this.addEventListener('install', function(event) {
  event.waitUntil(
    caches.create('v1').then(function(cache) {
     return cache.add(
        '/index.html',
        '/style.css',
        '/script.js',
        '/favicon.ico'
      );
    }, function(error) {
        console.error('error populating cache ' + error);
    };
  );
 });

So that the app can then respond to requests for resources when offline:

 this.addEventListener('fetch', function(event) {
  event.respondWith(
    caches.match(event.request).catch(function() {
      return event.default();
    })
  );
 });

The Times They Are A Changin’ (Open Web Remix)

In the run up to the “Mozlandia” work week in Portland, and in reflection of the last three years of the Firefox OS project, for a bit of fun I’ve reworked a Bob Dylan song to celebrate our incredible journey so far.

Here’s a video featuring some of my memories from the last three years, with Siobhan (my fiancée) and me singing the song at you! There are even lyrics so you can sing along 😉

“Keep on rockin’ the free web” — Potch

What is a Web App?

What is a web app? What is the difference between a web app and a web site? What is the difference between a web app and a non-web app?

In terms of User Experience there is a long continuum between “web site” and “web app” and the boundary between the two is not always clear. There are some characteristics that users perceive as being more “app like” and some as more “web like”.

The presence of web browser-like user interface elements like a URL bar and navigation controls are likely to make a user feel like they’re using a web site rather than an app for example, whereas content which appears to run independently of the browser feels more like an app. Apps are generally assumed to have at least limited functionality without an Internet connection and tend to have the concept of residing in a self-contained way on the local device after being “installed”, rather than being navigated to somewhere on the Internet.

From a technical point of view there is in fact usually very little difference between a web site and a web app. Different platforms currently deal with the concept of “web apps” in all sorts of different, incompatible ways, but very often the main difference between a web site and web app is simply the presence of an “app manifest”. The app manifest is a file containing a collection of metadata which is used when “installing” the app to create an icon on a homescreen or launcher.

At the moment pretty much every platform has its own proprietary app manifest format, but the W3C has the beginnings of a proposed specification for a standard “Manifest for web applications” which is starting to get traction with multiple browser vendors.

Web Manifest – Describing an App

Below is an example of a web app manifest following the proposed standard format.

http://example.com/myapp/manifest.json:

{
  "name": "My App",
  "icons": [{
    "src": "/myapp/icon.png",
    "sizes": "64x64",
    "type": "image/png"
  }],
  "start_url": "/myapp/index.html"
}

The manifest file is referenced inside the HTML of a web page using a link relation. This is cool because with this approach a web app doesn’t have to be distributed through a centrally controlled app store, it can be discovered and installed from any web page.

http://example.com/myapp/index.html:

<!DOCTYPE html>
<html>
  <head>
    <title>My App - Welcome</title>
    <link rel="manifest" href="manifest.json">
    <meta name="application-name" content="My App">
    <link rel="icon" sizes="64x64" href="icon.png">
...

As you can see from the example, these basic pieces of metadata which describe things like a name, an icon and a start URL are not that interesting in themselves because these things can already be expressed in HTML in a web standard way. But there are some other other proposed properties which could be much more interesting.

Display Modes – Breaking out of the Browser

We said above that one thing that makes a web app feel more app like is when it runs outside of the browser, without common browser UI elements like the URL bar and navigation controls. The proposed “display” property of the manifest allows authors of web content which is designed to function without the need for these UI elements to express that they want their content to run outside of the browser.

http://example.com/myapp/manifest.json:

{
  "name": "My App",
  "icons": [{
    "src": "/myapp/icon.png",
    "sizes": "64x64",
    "type": "image/png"
  }],
  "start_url": "/myapp/index.html"
  "scope": "/myapp"
  "display": "standalone"
}

The proposed display modes are “fullscreen”, “standalone”, “minimal-ui” and “browser”. The “browser” display mode opens the content in the user agent’s conventional method (e.g. a browser tab), but all of the other display modes open the content separate from the browser, with varying levels of browser UI.

There’s also a proposed “orientation” property which allows the content author to specify the default orientation (i.e. portrait/landscape) of their content.

App Scope – A Slice of the Web

In order for a web app to be treated separately from the rest of the web, we need to be able to define which parts of the web are part of the app, and which are not. The proposed “scope” property of the manifest defines the URL scope to which the manifest applies.

By default the scope of a web app is anything from the same origin as its manifest, but a single origin can also be sliced up into multiple apps or into app and non-app content.

Below is an example of a web app manifest with a defined scope.

http://example.com/myapp/manifest.json:

{
  "name": "My App",
  "icons": [{
    "src": "/myapp/icon.png",
    "sizes": "64x64",
    "type": "image/png"
  }],
  "start_url": "/myapp/index.html"
  "scope": "/myapp"
}

From the user’s point of view they can browse around the web, seamlessly navigating between web apps and web sites until they come across something they want to keep on their device and use often. They can then slice off that part of the web by “bookmarking” or “installing” it on their device to create an icon on their homescreen or launcher. From that point on, that slice of the web will be treated separately from the browser in its own “app”.

Without a defined scope, a web app is just a web page opened in a browser window which can then be navigated to any URL. If that window doesn’t have any browser-like navigation controls or a URL bar then the user can get stranded at a dead on the web with no way to go back, or worse still can be fooled into thinking that a web page they thought was part of a web app they trust is actually from another, malicious, origin.

The web browser is like a catch-all app for browsing all of the parts of the web which the user hasn’t sliced off to use as a standalone app. Once a web app is registered with the user agent as managing a defined slice of the web, the user can seamlessly link into and out of installed web apps and the rest of the web as they please.

Service Workers – Going Offline

We said above that another characteristic users often associate with “apps” is their ability to work offline, in the absence of a connection to the Internet. This is historically something the web has done pretty badly at. AppCache was a proposed standard intended for this purpose, but there are many common problems and limitations of that technology which make it difficult or impractical to use in many cases.

A new, much more versatile, proposed standard is called Service Workers. Service Workers allow a script to be registered as managing a slice of the web, even whilst offline, by intercepting HTTP requests to URLs within a specified scope. A Service Worker can keep an offline cache of web resources and decide when to use the offline version and when to fetch a new version over the Internet.

The programmable nature of Service Workers make them an extremely versatile tool in adding app-like capabilities to web content and getting rid of the notion that using the web requires a persistent connection to the Internet. Service Workers have lots of support from multiple browser vendors and you can expect to see them coming to life soon.

The proposed “service_worker” property of the manifest allows a content author to define a Service Worker which should be registered with a specified URL scope when a web app is installed or bookmarked on a device. That means that in the process of installing a web app, an offline cache of web resources can be populated and other installation steps can take place.

Below is our example web app manifest with a Service Worker defined.

http://example.com/myapp/manifest.json:

{
  "name": "My App",
  "icons": [{
    "src": "/myapp/icon.png",
    "sizes": "64x64",
    "type": "image/png"
  }],
  "start_url": "/myapp/index.html"
  "scope": "/myapp"
  "service_worker": {
    "src": "app.js",
    "scope": "/myapp"
  }
}

Packages – The Good, the Bad and the Ugly

There’s a whole category of apps which many people refer to as “web apps” but which are delivered as a package of resources to be downloaded and installed locally on a device, separate from the web. Although these resources may use web technologies like HTML, CSS and Javascript, if those resources are not associated with real URLs on the web, then in my view they are by definition not part of a web app.

The reason this approach is commonly taken is that it allows operating system developers and content authors to side-step some of the current shortcomings of the web platform. Packaging all of the resources of an app into a single file which can be downloaded and installed on a device is the simplest way to solve the offline problem. It also has the convenience that the contents of that package can easily be reviewed and cryptographically signed by a trusted party in order to safely give the app privileged access to system functions which would currently be unsafe to expose to the web.

Unfortunately the packaged app approach misses out on many of the biggest benefits of the web, like its universal and inter-linked nature. You can’t hyperlink into a packaged app, and providing an updated version of the app requires a completely different mechanism to that of web content.

We have seen above how Service Workers hold some promise in finally solving the offline problem, but packages as a concept may still have some value on the web. The proposed “Packaging on the Web” specification is exploring ways to take advantage of some of the benefits of packages, whilst retaining all the benefits of URLs and the web.

This specification does not explore a new security model for exposing more privileged APIs to the web however, which in my view is the single biggest unsolved problem we now have left on the web as a platform.

Conclusions

In conclusion, a look at some of the latest emerging web standards tells us that the answer to the question “what is a web app?” is that a web app is simply a slice of the web which can be used separately from the browser.

With that in mind, web authors should design their content to work just as well inside and outside the browser and just as well offline as online.

Packaged apps are not web apps and are always a platform-specific solution. They should only be considered as a last resort for apps which need access to privileged functionality that can’t yet be safely exposed to the web. New web technologies will help negate the need for packages for offline functionality, but packages as a concept may still have a role on the web. A security model suitable for exposing more privileged functionality to the web is one of the last remaining unsolved challenges for the web as a platform.

The web is the biggest ecosystem of content that exists, far bigger than any proprietary walled garden of curated content. Lots of cool stuff is possible using web technologies to build experiences which users would consider “app like”, but creating a great user experience on the web doesn’t require replicating all of the other trappings of proprietary apps. The web has lots of unique benefits over other app platforms and is unrivalled in its scale, ubiquity, openness and diversity.

It’s important that as we invent cool new web technologies we remember to agree on standards for them which work cross-platform, so that we don’t miss out on these unique benefits.

The web as a platform is here to stay!

Building the Firefox Browser for Firefox OS

Re-posted from Mozilla Hacks.

Boot to Gecko

As soon as the Boot to Gecko (B2G) project was announced in July 2011 I knew it something I wanted to contribute to. I’d already been working on the idea of a browser based OS for a while but it seemed Mozilla had the people, the technology and the influence to build something truly disruptive.

At the time Mozilla weren’t actively recruiting people to work on B2G, the team still only consisted of the four co-founders and the project was little more than an empty GitHub repository. But I got in touch the day after the announcement and after conversations with Chris, Andreas and Mike over Skype and a brief visit to Silicon Valley, I somehow managed to convince them to take me on (initially as a contractor) so I could work on the project full time.

A Web Browser Built from Web Technologies

On my first day Chris Jones told me “The next, highest-priority project is a very basic web browser, just a URL bar and back button basically.”

chris_jones

Chris and his bitesize browser, Taipei, December 2011

The team was creating a prototype smartphone user interface codenamed “Gaia”, built entirely with web technologies. Partly to prove it could be done, but partly to find the holes in the web platform that made it difficult and fill those holes with new Web APIs. I was asked to work on the first prototypes of a browser app, a camera app and a gallery app to help find some of those holes.

You might wonder why a browser-based OS needs a browser app at all, but the thinking for this prototype was that if other smartphone platforms had a browser app, then B2G would need one too.

The user interface of the desktop version of Firefox is written in highly privileged “chrome” code using the XUL markup language. On B2G it would need to be written in “content” using nothing but HTML, CSS and JavaScript, just like all the other apps. That would present some interesting challenges.

In the beginning, there was an <iframe>

It all started with a humble iframe, a text input for the URL bar and a go button, in fact you can see the first commit here. When you clicked the go button, it set the src attribute of the iframe to the contents of the text input, which caused the iframe to load the web page at that URL.

first_commit

First commit, November 2011

The first problem with trying to build a web browser using an iframe is that the same-origin policy in JavaScript prevents you accessing just about any information about what’s going on inside it if the content comes from a different origin than the browser itself. In particular, it’s not possible to access the contentWindow property and all of the information that gives access to. This policy exists for good reasons so in order to build a fully functional web browser we would have to figure out a way for a privileged web app to safely poke holes in that cross-origin boundary to get just enough information to do its job, but without creating serious security vulnerabilities or compromising the user’s privacy.

Another problem we came across quite quickly was that many web authors will go to great lengths to prevent their web site being loaded inside an iframe in order to prevent phishing attacks. A web server can send an X-Frame-Options HTTP response header instructing a user agent to simply not render the content, and there are also a variety of techniques for “framebusting” where a web site will actively try to break out of an iframe and load itself in the parent frame instead.

It was quickly obvious that we weren’t going to get very far building a web browser using web technologies without evolving the web technologies themselves.

The Browser API

I met Justin Lebar at the first B2G work week in Taipei in December 2011. He was tasked with modifying Gecko to make the browser app on Boot to Gecko possible. To me Gecko was (and largely still is) a giant black box of magic spells which take the code I write and turn it into dancing images on the screen. I needed a wizard who had a grasp on some of these spells, including a particularly strong spell called Docshell which only the most practised of wizards dare peer into.

justin

Justin at the first B2G Work Week in Taipei, December 2011

When I told Justin what I needed he made the kinds of sounds a mechanic makes when you take your car in for what you think is a simple problem but turns out costing the price of a new car. Justin had a better idea than I did as to what was needed, but I don’t think either of us realised the full scale of the task at hand.

With the adding of a simple boolean “mozbrowser” attribute to the HTML iframe element in Gecko, the Browser API was born. I tried adding features to the browser app and every time I found something that wasn’t possible with current web technologies, I went back to Justin to get him to cast a new magic spell.

There were easier approaches we could have taken to build the browser app. We could have added a mechanism to allow the browser to inject scripts into the iframe and communicate freely with the content inside, but we wanted to provide a safe API which anyone could use to build their own browser app and this approach would be too risky. So instead we built an explicit privileged API into the DOM to create a new class of iframe which could one day become a new standard HTML tag.

Keeping the Web Contained

The first thing we did was to try to trick web pages loaded inside an iframe into thinking they were not in fact inside an iframe. At first we had a crude solution which just ignored X-Frame-Options headers for iframes in whitelisted domains that had the mozbrowser attribute. That’s when we discovered that some web sites are quite clever at busting out of iframes. In the end we had to take other measures like making sure window.top pointed at the iframe rather than its parent so a web site couldn’t detect that it had a parent, and eventually also run every browser tab in its own system process to completely isolate them from each other.

Once we had the animal that is the web contained, we needed to poke a few air holes to let it breathe. There’s some information we need to let out of the iframe in the form of events: when the location, title or icon of a web page changes (locationchange, titlechange and iconchange); when a page starts and finishes loading (loadstart, loadend) and when the security characteristics of the currently loaded page changes (securitychange). This all allows us to keep the address bar and title bar up to date and show a progress indicator.

The browser app needs to be able to navigate the iframe by telling it to goBack(), goForward(), stop() and reload(). We also need to be able to explicitly ask for information like characteristics of the session history (getCanGoBack(), getCanGoForward()) to determine which navigation buttons to display.

With these basics in place it was possible to build a simple functional browser app.

The Prototype

The Gaia project’s first UX designer was Josh Carpenter. At an intensive work week in Paris the week before Mobile World Congress in February 2012, Josh created UI mockups for all the basic features of a smartphone, including a simple browser, and we built a prototype to those designs.

josh

Josh and me plotting over a beer in Paris.

 

The prototype browser app could navigate web content, keep it contained and display basic information about the content being viewed. This would be the version demonstrated at MWC in Barcelona that year.

mwc_browser

Simple browser demo for Mobile World Congress, February 2012

Building a Team

At a work week in Qualcomm’s offices in San Diego in May 2012 I was able to give a demo of a slightly more advanced basic browser web app running inside Firefox on the desktop. But it was still very basic. We needed a team to start building something good enough that we could ship it on real devices.

browser_inception

“Browser Inception”, San Diego May 2012

San Diego was also where I first met Dale Harvey, a brave Scotsman who came on board to help with Gaia. His first port of call was to help out with the browser app.

dale

Dale Getting on Board in San Diego, May 2012

One of the first things Dale worked on was creating multiple tabs in the browser and even adding a screenshotting spell to the Browser API to show thumbnails of browser tabs (I told you he was brave).

By this time we had also started to borrow Larissa Co, a brilliant designer from the Firefox team, to work on the interaction design and Patryk Adamczyk, formerly of RIM, to work on the visual design for the browser on B2G. That was when it started to look more like a Firefox browser.

start_page

Early UI Mockup, July 2012

Things that Pop Up

Web pages like to make things pop up. For a start they like to alert(), prompt() or confirm() things with you. Sometimes they like to open() a new browser window (and close() them again), open a link in a _blank window, ask you for a password, ask for your permission to do something, ask you to select an option from a menu, open a context menu or confirm re-sending the contents of a form.

alert

An alert(), version 1.0

All of this required new events in the Browser API, which meant more spells for Justin to cast.

Scroll, Pan and Zoom

Moving around web pages on web devices works a little differently from on the desktop. Rather than scroll bars or a scroll wheel on a mouse it uses touch input and a system called Asynchronous Pan and Zoom to allow the user to pan around a web page by dragging it and scrolling it using “kinetic scrolling” which feels like it has some physics to it.

The first implementation of kinetic scrolling was written in JavaScript by Frenchman and Gaia leader Vivien Nicolas, specifically for Gaia, but it would later be written in a cross-platform way in Gecko to unify the code used on B2G and Android.

One of the trickier interactions to get right was that we wanted the address bar to hide as you scrolled down the page in order to make more room for content, then show again when you scroll back to the top of the page.

This required adding asyncscroll events which tapped directly into the Asynchronous Pan and Zoom code so that the browser knew not only when the user directly manipulated the page, but how much it scrolled based on physics, asynchronously from the user’s interaction.

Storing Stuff

One of the most loved features of Firefox is the “Awesomebar”, a combined address bar, search bar (and on mobile, title bar) which lets you quickly get to the content you’re looking for. You type a few characters and immediately start to see matching web pages from your browsing history, ranked by a “frecency” algorithm.

On the desktop and on Android all of this data is stored in the “Places” database as part of privileged “chrome” code. In order to implement this feature in B2G we would need to use the local storage capabilities of the web, and for that we chose IndexedDB. We built a Places database in IndexedDB which would store all of the “places” a user visits on the web including their URL, title and icon, and store all the times the user visited that page. It would also be used to store the users bookmarks and rank top sites by “frecency”.

awesomebar

Awesomebar, version 1.0

Clearing Stuff

As you browse around the web Gecko also stores a bunch of data about the places you’ve been. That can be cookies, offline pages, localStorage, IndexedDB databases and all sorts of other bits of data. Firefox browsers provide a way for you to clear all of this data, so methods needed to be added to the Browser API to allow this data to be cleared from the browser settings in B2G.

settings

Browser settings, version 1.0

Handling Crashes

Sometimes web pages crash the browser. In B2G every web app and every browser tab runs in its own system process so that should the worst happen, it will only cause that one window/tab to crash. In fact, due to the memory constraints of the low-end smartphones B2G would initially target, sometimes the system will intentionally kill a background app or browser tab in order to conserve memory. The browser app needs to be informed when this happens and needs to be able to recover seamlessly so that in most cases the user doesn’t even realise a process was killed. Events were added to the Browser API for this purpose.

crash

Crashed tab, version 1.0

Talking to Other Apps

Common use cases of a mobile browser are for the user to want to share a URL using another app like a social networking tool, or for another app to want to view a URL using the browser.

B2G implemented Web Activities for this purpose, to add a capability to the web for apps to interact with each other, but in an app-agnostic way. So for example the user can click on a share button in the browser app and B2G will fire a “share URL” Web Activity which can then be handled by any installed app which has registered to handle that type of Web Activity.

share

Share Web Activity, version 1.2

Working Offline

Despite the fact that B2G and Gaia are built on the web, it is a requirement that all of the built-in Gaia apps should be able to function offline, when an Internet connection is unavailable or patchy, so that the user can still make phone calls, take photos and listen to music etc.. At first we started to use AppCache for this purpose, which was the web’s first attempt at making web apps work offline. Unfortunately we soon ran into many of the common problems and limitations of that technology and found it didn’t fulfill all of our requirements.

In order to ship version 1.0 of B2G on time, we were forced to implement “packaged apps” to fulfill all of the offline and security requirements for built-in Gaia apps. Packaged apps solved our problems but they are not truly web apps because they don’t have a real URL on the Internet, and attempts to standardise them didn’t get much traction. Packaged apps were intended very much as a temporary solution and we are working hard at adding new capabilities like ServiceWorkers, standardised hosted packages and manifests to the web so that eventually proprietary packaged apps won’t be necessary for a full offline experience.

offline

Offline, version 1.4

Spit and Polish

Finally we applied a good deal of spit and polish to the browser app UI to make it clean and fluid to use, making full use of hardware-accelerated CSS animations, and a sprinkling of Firefoxy interaction and visual design to make the youngest member of the Firefox browser family feel consistent with its brothers and sisters on other platforms.

Shipping 1.0

At an epic work week in Berlin in January 2013 hosted by Deutsche Telekom the whole B2G team, including engineers from multiple competing mobile networks and device manufacturers, got together with the common cause of shipping B2G 1.0, in time to demo at Mobile World Congress in Barcelona in February. The team sprinted towards this goal by fixing an incredible 200 bugs in one week.

berlin

Version 1.0 Team, Berlin Work Week, January 2013

In the last few minutes of the week Andreas Gal excitedly declared “Zarro Gaia Boogs”, signifying version 1.0 of Gaia was complete, with the rest of B2G to shortly follow over the weekend. Within around 18 months a dedicated team spanning multiple organisations had come together working entirely in the open to turn an empty GitHub repository into a fully functioning mobile operating system which would later ship on real devices as Firefox OS 1.0.1.

zarro_boogs

Zarro Gaia Boogs, January 2013

v1

Browser app v1.0

So having attended Mobile World Congress 2012 with a prototype and a promise to deliver commercial devices into the market, we were able to return in 2013 having delivered on that promise by fully launching the “Firefox OS” brand with multiple devices on multiple mobile networks with a launch that really stole the show at the biggest mobile conference in the world. Firefox OS had arrived.

mwc

Mobile World Congress, Barcelona, February 2013

1.x

Firefox OS 1.1 quickly followed and by the time we started working on version 1.2 the project had grown significantly. We re-organised into autonomous agile teams focused on product areas, the browser app being one. That meant we now had a dedicated team with designers, engineers, a test engineer, a product manager and a project manager.

browser_team

The browser team, London work week, July 2013

Firefox OS moved to a rapid release “train model” of development like Firefox, where a new version is delivered every 12 weeks. We quickly added new features and worked on improving performance to get the best out of the low end hardware we were shipping on in emerging markets.

v1.4

Browser app v1.4

“Haida”

Version 1.0 of Firefox OS was very much about proving that we could build what already exists on other smartphones, but entirely using open web technologies. That included a browser app.

Once we’d proved that was possible and put real devices on shelves in the market it was time to figure out what would differentiate Firefox OS as a product going forward. We wanted to build something that doesn’t just imitate what’s already been done, but which plays to the unique strengths of the web to build something that’s true to Mozilla’s DNA, is the best way to experience the web, and is the platform that HTML5 deserves.

Below is a mockup I created right back towards the start of the project at the end of 2011, before we even had a UX team. I mentioned earlier that the Awesomebar is a core part of the Firefox experience in Firefox browsers. My proposal back then was to build a system-wide Awesomebar which could search the whole device, including your apps and their contents, and be accessible from anywhere in the OS.

b2g_awseomescreen

Very early mockup of a system-wide Awesomebar, December 2011

At the time, this was considered a little too radical for version 1.0 and our focus really needed to be on innovating in the web technology needed to build a mobile OS, not necessarily the UX. We would instead take a more conservative approach to the user interface design and build a browser app a lot like the one we’d built for Android.

In practice that meant that we in fact built two browsers in Firefox OS. One was the browser app which managed the world of “web sites” and the other was the window manager in the system app which managed the world of “web apps” .

In reality on the web there isn’t so much of a distinction between web apps and web sites – each exists on a long continuum of user experience with a very blurry boundary in the middle.

In March 2013, with Firefox OS 1.0 out of the door, Josh Carpenter put me in touch with Gordon Brander, a member of the UX team who had been thinking along the same lines as me. In fact Gordon being as much of an engineer as he is a designer, had gone as far as to write a basic prototype in JavaScript.

rocketbar_prototype

Gordon’s Rocketbar Prototype, March 2013

Gordon and I started to meet weekly to discuss the concept he had by then codenamed “Rocketbar”, but it was a bit of a side project with a few interested people.

In April 2013 the UX team had a summit in London where they got together to discuss future directions for the user experience of Firefox OS. I was lucky enough to be invited along to not only observe but participate in this process, Josh being keen to maintain a close collaboration between Design and Engineering.

We brainstormed around what was unique about the experience of the web and how we might create a unique user experience which played to those strengths. A big focus was on “flow”, the way that we can meander through the web by following hyperlinks. The web isn’t a world of monolithic apps with clear boundaries between them, it is an experience of surfing from one web site to another, flowing through content.

ux_workshop

Brainstorming session, London, April 2013

In the coming weeks the UX team would create some early designs for a concept (eventually codenamed “Haida”) which would blur the lines between web apps and web sites and create a unique user experience which flows like the web does. This would eventually include not only the “Rocketbar”, which would be accessible across the whole OS and seamlessly adapt to different types of web content, but also “sheets”, which would split single page web apps into multiple pages which you could swipe through with intuitive edge gestures. It would also eventually include a content model based around live apps which you can surf to, use, and then bookmark if you choose to, rather than monolithic apps which you have to install from a central app store before you can use them.

In June 2013 a small group of designers and engineers met in Paris to develop a throwaway prototype of Haida, to rapidly iterate on some of the more radical concepts and put them through user testing.

haida_team

Haida Prototyping, Paris, June 2013

josh_and_gordon

Josh and Gordon working in a highly co-ordinated fashion, Paris, June 2013

haida_engineers

Wizards at work, Paris, June 2013

2.x and the Future

Fast forward to the present and the browser team has been merged into the “Systems Front End” team. The results of the Haida prototyping and user testing are slowly starting to make their way into the main Firefox OS product. It won’t happen all at once, but it will happen in small pieces as we iterate and learn.

In version 2.0 of Firefox OS the homescreen search feature from 1.x will be replaced with a new search experience developed in conjunction with a new homescreen, implemented by Kevin Grandon, which will lay the foundations for “Rocketbar”. In version 2.1 our intention is to completely merge the browser app into the system app so that browser tabs become “sheets” alongside apps in the task manager and the “Rocketbar” is accessible from anywhere in the OS. The Rocketbar will adapt to different types of web content and shrink down into the status bar when not in use. Edge gestures will allow you to swipe between web apps and browser windows and eventually apps will be able to spawn multiple sheets.

rocketbar

UI Mockups of Rocketbar in expanded and collapsed state, July 2014

In parallel we see the evolution of web standards around manifests, packages and webviews and ongoing discussions around what defines the scope of an “app”.

Version 1.x of Firefox OS was built with web technologies but still has quite a similar user experience to other mobile platforms when it comes to installing and using apps, and browsing the web. Going forward I think you can expect to see the DNA of the web come through into the user interface with a unified experience which breaks down the barriers between web apps and web sites, allowing you to freely flow between the two.

Firefox OS is an open source project developed completely in the open. If you’re interested in contributing to Gaia, take a look at the “Developing Gaia” page on MDN. If you’re interested in creating your own HTML5 app to run on Firefox OS take a look at the “App Center“.

Introducing DRD Pi

I’m trying to build a Raspberry Pi powered robot based on the DRDs from Farscape, I thought I’d blog my progress.

DRD

DRDs or “Diagnostic Repair Drones” are robots from the cult science fiction series Farscape. They carry out various functions aboard a leviathian (a species of living biomechanoid spaceship) including repairing and maintaining the ship. They’re ovoid in shape and they have two moving eye stalks and all sorts of tools like a robotic claw and a plasma welder.

DRD

Here’s some video footage from the series to give you an idea of what these little guys get up to:

DRD Kit

The original DRDs were designed and built by the extremely talented folks at the Jim Henson Creature Shop in London (yes Jim Henson as in the Muppets!). They built lots of different variations of the robot over the years to be used in shooting different scenes for the show, but to my knowledge they’ve never released any designs.

I assumed I was going to have to painstakingly design a 3D computer model of one based on frame grabs from my DVDs of the series. I then planned to track down someone with a CNC router and a vacuum forming machine and persuade them to let me use them. Either that or find someone with an industrial sized 3D printer!

Luckily I came across a special effects company in the US who sells a kit to build a model of a DRD. The model is made from hollow cast fiberglass and resin and comes with ribbed plastic for the eye stalks, eye pieces with clear lenses, two parts of a claw and some colourful wires to make it look the part.

drd_kit

The kit isn’t perfect. The size, shape and proportions aren’t quite right and the finish is a bit rough but it’s good enough for my purposes. The part I’m really interested in is the robotics so I’m grateful that someone has already done the work for me on the basic shell.

The web site provides video tutorials on how to build the model and then how to put LEDs in the eyes and mount an remote controlled car underneath to make it move about in a bit of a crude fashion.

We can be a bit more sophisticated than that.

Raspberry Pi

The Raspberry Pi is a credit-card sized computer developed in the UK by the not-for-profit Raspberry Pi Foundation to promote the teaching of programming in schools. It’s a single-board computer with a 700Mhz ARM processor and 512MB RAM, boots off an SD card and costs only around £30.

This is my Raspberry Pi:

raspberry_pi

Gertboard

The Gertboard is an expansion board which attaches to the Raspberry Pi via its GPIO pins and helps when experimenting with interfacing the Pi with the outside world. It comes with an Arduino compatible AVR microcontroller, analogue to digital converters, digital to analogue converters, a motor controller, push buttons, LEDs and much more.

gertboard

Booting the Pi

The Raspberry Pi can boot Linux from an SD card and the most popular distribution is Raspbian which is a Debian-derivative. You can download an image and flash it to an SD card, or even buy an SD card with it already loaded.

To boot the Raspberry Pi all you need to do is insert your Raspbian SD card, plug it into a TV via either the HDMI port or the composite video port and power it up by plugging it into a Micro USB phone charger.

Here’s my Raspberry Pi booted and plugged into an old CRT TV:

raspberry_pi_and_tv

 

Logging In Remotely

It’s cool that I can plug the Raspberry Pi into a TV, but I don’t want to be squinting at an old portable TV or sitting in the lounge next to my big flatscreen TV all the time I’m programming the robot, so I want to be able to log in remotely. Also, my plan is to build a web interface to control the robot over WiFi, so it’s going to need to connect to a network at some point.

First I plugged a USB keyboard into the Raspberry Pi and an ethernet cable to connect it to my network. The SSH daemon is already started by default, but I wanted to set a static IP address so that I always knew what to log in to.

I logged into the Raspberry Pi locally (the default username is pi and the password is raspberry) and edited the network configuration using the vi text editor.

$ sudo vi /etc/network/interfaces

I provided the following configuration to assign a static IP address of 192.168.1.42 on my local network:

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
address 192.168.1.42
netmask 255.255.255.0
gateway 192.168.1.1

Then restart the network interface with:
$ sudo ifdown -a
$ sudo ifup -a

Then check that I’m connected to the network, and the Internet by pinging Google.

$ ping google.com

I see that I’m successfully connected, so I can now log into the Raspberry Pi remotely using its new static IP.

From my desktop Linux box I type:

$ ssh pi@192.168.1.42

type in the password “raspberry”, and voilà! I’m logged in.

ssh

What’s Next

I hope you weren’t expecting to see a finished robot! There’s a very long way to go yet.

If you desperately wanted to see a finished robot, here’s a picture of the last one I worked on, a line following robot we built at university powered by a PIC microcontroller.

BEAST

Next I want to start playing around with the Gertboard and and make LEDs blink on and off from Python.