We’ve Moved!

Please give a warm welcome to the new and improved Symbian Multimedia Blog!

The new blog replaces this one.  This blog and all its posts will remain online so as not to break any existing links or bookmarks, however no new content will be posted here.  Instead new posts will be appearing on the new blog from now on. We therefore suggest updating your bookmarks and feed subscriptions accordingly.

As the URL suggests, this blog started life as a place for myself (the Symbian Foundation’s Multimedia technology manager) to blog on the topic of multimedia. However, over time new authors such as Jeremy and James joined and it transformed into a more general Symbian Multimedia blog. In order to avoid future confusion and improve visibility of this blog we decided to relocate to a more descriptive URL and that is how the new blog came to be!

We hope you enjoy our future posts and look forward to reading your comments!

Following the ECAM Contribution

A while back, on the main Symbian blog, I announced that ISB were making a contribution that would enable camera-based use-cases in QEMU.  Essentially the contribution would allow a webcam to be used on the host PC and to function like a camera on a phone.

To keep everyone updated on the progress of this contribution, I have created a wiki page.  Please drop by, have a read, comment on the project and perhaps give some thought to getting involved!  The contribution is initially focused on providing a complete vertical stack, but there is some horizontal diversification that would be of benefit.  For example, to add support for additional brands of webcam, or support more advanced use-cases like scene modes.

Collaborative Test Database

One of the most exciting things about working for Symbian is that its model enables a wider variety of contributions than source code.  Contributions of all forms are valuable, including words (in the form of discussions in the various forums and Ideas site etc) but also test effort.

There are many areas of platform testing that are resource-intensive and common between companies, for example interoperability testing and multimedia playability testing.  Therefore one of the specific ideas we have been working towards is a web database to enable community memebers to coordinate testing efforts and collaborate.  This will help reduce R&D time and costs for everyone.

The database is simple enough.  The community can contribute test definitions, and the database will sort these to prioritise the least-recently-run tests by showing these first.  Then community members can contribute test time by running the test and recording the results, as either a pass or Bugzilla entries for failures.

Various spins of the database are possible.  The image below shows a mock-up of one view – a playability testing database (see also this wiki page).  Other spins include a plug-fest database for maximising hardware interoperability testing (such as Bluetooth devices, or HDMI), or a multimedia device-to-device interoperability database for testing content generated on one device can be played back on another.

Of course, it’s not perfect – the playability database especially needs careful integration with existing test strategies to actually realise the cost savings (the benefits from the other spins are much easier to realise). 

However, by working together as a community these ideas can be rapidly improved, so please feel free to add comments here or on the wiki page.  We are also hopeful that very soon we can announce a contribution by one of our members of this database!

Calling all Developers Part 2

Development / Contribution Opportunities for improving Browser/Player Integration

 

 

Following on from yesterday’s post, I wanted to start a discussion about the ways in which the browser and the player could be more closely integrated, so as to improve the overall streaming video experience for end-users.  Here are some of the ideas that myself and the two companies shown above came up with – please feel free to add to the list!

• Enabling the player to jump to another URL on finishing playback: the use case is where a user receives a message or alert which links to a video.  Once the video has finished, the browser goes to a page to enable the user to learn more about what they have watched
• Bandwidth detection in the device: enables the player to tell the server which stream quality to use based on the player’s knowledge of available bandwidth (e.g. 2G vs. 3G vs. WiFi bearer)
• Phone can seamlessly integrate playback of a number of streams: for example, this could enable a news summary built from individual stories based on user preferences.  Or a service could feed in adverts at appropriate points in the stream.
• Player awareness of when in an advert sequence: so transport controls can be disabled (enables easier monetisation of content, and hence better content for users)
• Video hot-spots: the server can indicate hoyspot time offsets to the device, which can be used to trigger UI events.  For example, an initial summary of a news story could be expanded on based on some UI feedback, or a user could request more info following an advert (which would queue up a web page for viewing later)

These ideas are quite focused on hooks to improve services.  While I agree there are many improvements needed to the direct end-user experience, I strongly believe it is important for us in the device community to work closely with service providers, since providing a better platform for the guys who build video services ultimately leads to better products for end-users.

So – do you fancy a go at building these improvements?  Get in touch!

Calling all Developers!

Improving browser / player integration

Following on from my earlier post about the role of mobile in the lean forward / lean back worlds, there are a number of ways in which browser / video player integration can be improved and streamlined.

Plenty of you I’m sure will have ideas here.  A few of us in the community have been discussing this already, and a forthcoming blog post will outline these ideas.  If you have any of your own, please feel free to feed them back as a comment.

Many of these improvements rely on experimenting with new ways in which the device and the server can interact – for example to feed back information about bandwidth availability, which the server can use accordingly. 

The key problem here is having a “real” environment in which to test ideas – in other words, a real streaming server.  Fortunately Yospace have helped out here by providing a demo environment on their YospaceCDS product.  So let’s think about exciting new things we can do to enhance mobile video streaming, and get developing!

To see the environment, point your phone’s browser at http://cds1-feed.yospace.com/00002544283.  Click “View” on the first screen, then either “Play (Standard)” or “Play (Medium)” on the next screen.

Note: I’d like to keep the comments to this post free for brainstorming ideas.  If you are having problems getting the demo to work, I’ve created a thread in the multimedia forum.

Getting the Community’s View: Yospace

One of the great benefits of working with the Symbian community is that it is much easier to share views with a wider variety of people and companies.  Breadth of views will make for much more varied and interesting contributions, and result in more exciting roadmaps for everyone.  In this first of a series of postings, I met up with David Springall, CTO of Yospace, to discuss mobile video.

 

 

What is Yospace’s role in the video delivery ecosystem?

We provide a “cloud-based” service that deals with the problems of getting online video out to mobile handsets.  With the multitude of handsets professing to support video – each with their own set of idiosyncrasies – getting the optimum user experience (and in some cases, *any* user experience!) for the widest possible audience is technically very challenging.  We’ve wrapped up our 4 years of experience of running our own consumer facing video site and made this technology available as software-as-a-service solution. 

What are the main blockers to video delivery services on mobiles becoming mass market?

It was quite fashionable a few years ago to think that you had to target video content for mobile.  In the last few years we’ve witnessed an explosion of short form content being produced for the web and that usually works perfectly well in the mobile medium.  So, the issue is not availability of video content, it’s the fact that it’s perceived as quite expensive build a service that gets video out to mobile coupled with the fact that the audiences still relatively small.  For most publishers the perceived cost of getting video out to mobile combined with the fact that the advertising revenues reflect the smaller audience has taken a mobile video offering off of the table.   Operator data tariffs are making it more and more affordable to access rich media services, the devices are out there, but the publishers need to build their audiences.  We believe we are helping break this catch-22 situation by making it really easy and affordable for publishers to start converging their rich online offering with mobile.

What have you found easy about working on the Symbian platform?

Our experience with the Symbian platform has been extremely positive – stuff usually just works.  Most of our time is spent working around handset foibles and when it comes to video streaming and playback the Symbian devices seem to be the least fussy.  For example, if the device can’t negotiate a UDP stream for RTP packets, it just gets on with it over TCP.  If everything out there worked as well, then perhaps we wouldn’t be needed!

What are the features you’d like to see from the Symbian multimedia world?

We would love to have more control over the in-built video player without the need to install software onto the phone.  The primary motivation is to be able to offer a richer medium for advertisers, which is where we predict most of the funding of mobile video will come from.  Ideally we would like to see some means by which it would be possible to offer interactivity with the video — so that advertising inserted into the video — as a pre-roll, or even an overlay — could be ‘clicked through’ somehow. 

It would be great if the in-built video player when streaming could offer bandwidth detection.  The RTSP protocol has provisions for this so it would be great to see it implemented on the handset side.  If this worked, I think that Symbian would be able to offer a superior playback user experience because the RTSP/RTP protocol it supports is very network efficient.

How do you see mobile video delivery evolving over the coming years?

We are likely to see a significant amount of interest from content providers over the next two years as we all get used to enjoying the benefits of internet and mobile convergence.  It’s really important that the source of revenue for these content providers, which I believe will largely come from sponsorship and advertising needs to be supported by the technology.  Developers will find their way around any shortcomings in the native video playback capabilities for the purposes of supporting their paymasters better and that may result in a less than ideal user experience which may have a negative effect on user uptake, which in turn is critical for the investment by content providers to stack up.

Lean Forward, Lean Back: Mobile Multimedia in the Three-Screen World

One topic creating buzz in the mobile world at the minute is the “Three-Screen Strategy”, in which services present a unified product across the three screens of the TV, phone and PC.

During last week’s Streaming Media Europe event, this was one of the key themes of the show, and also the topic of a keynote by Myles Macbean of Disney. The key issue for me during this keynote was the observation of how the Web 2.0 generation actively multitasks when connected to the Internet. For example, whilst at the PC, a browsing session could launch games and video content, with attention being diverted by live chat sessions, all whilst supposedly doing homework.

Everyone is used to multitasking on the PC, but Symbian is one of the few phone platforms that supports it (and has done for many years). This allows you to do many new things, for example, browse the web whilst listening to music over Bluetooth headsets, all whilst your phone’s GPS is tracking your location and giving directions!

So how can Symbian help enable the view of web usage that Myles described? The key issue is the difference between the so-called “lean forward” and “lean back” experiences of multimedia, which refers to the difference in how people engage with the PC (lean forward, more interactive) vs. the TV (lean back, more emphasis on linear content consumption).

The mobile sits squarely in the middle here, since users can be engaged in both a “lean forward”, interactive way and a “lean backward”, linear way. I think there are several implications of this:

• Architecturally this presents challenges – devices need both an optimised, high quality media path for consuming linear content (hardware decoding, picture quality improvement algorithms, no frame drops allowed) and a flexible path for lean forward content (support multiple parallel decodes at a lower quality). The openness of Symbian means devices can support both models, but we in the community need to keep both in mind as we evolve the platform, so as to avoid constraining either mode
• Critically, mobile use-cases allow users to transition quickly between these two modes – for example, browsing movie trailers (lean forward) can lead to a movie purchase (again, lean forward), but then immediately into watching the movie (lean back). This could require the ability to hand over content between the two modes, for example dropping out of the optimised path to the flexible path when an Instant Messaging session starts (with the video minimising into a corner)
• Lean-forward is typically a personalised environment, while lead-back is typically uniform for all users. The phone allows much more cross-over between the two worlds – for example, linear consumption of a TV news channel could transition into pushed updates of stories in which the user is particularly interested.

I’d be interested in your thoughts on this, especially if it leads to ideas for future projects!

(footnote: of course, I understand that for many people, the PC is the main source of lean-back entertainment in the home. However, I believe that these users are in the minority overall, even if they may make up a significant proportion of a technical community such as Symbian. As a result, I think the phone offers the more attractive route to linking the lean-forward and lean-back worlds).

See you at SEE

Multimedia Birds of a Feather flock together!

As Jezar posted in the Multimedia forum: There will be a Multimedia Birds of a Feather (BoF) session at this year’s Symbian Exchange & Exposition. This is intended to be an informal gathering of developers and anyone other interested parties around the topic of multimedia in Symbian.

The details are:

• 27. October (Day 1)
• 15:00 – 16:00

• Area 2 (in the Main Stage area)
  

(Follow the red line on the floor plan on the right to find the main stage area where all the BoFs are)

The current mm package owner Jezar, the foundation’s multimedia technology manager (and owner of this blog) Martin Webb and myself will all be attending and happy to discuss all things multimedia!

So, whether you’re:

• a System Integrator
• a Silicon Vendor
• a Codec Licensor
• a Network Operator
• an Application Developer

…or indeed anyone else, we’d love to have a chat!

Now that Symbian OS is open source, the future of multimedia is very much in your hands:

• Perhaps there is an application that you want to write for the platform, but don’t know how to get the performance you’re looking for.
• Maybe you’ve developed a super-efficient codec, and want to know how to add it as a system-wide resource to demonstrate to your clients

Whatever your interest in multimedia, please come and visit us and talk to us about your project. We’d like to engage with the community as much as possible so that we can build into the platform the requirements that matter most.

Hope to see you there!

A Surprise Move

These days there really isn’t – it has to be said – much that surprises me in this business. I’ve seen some very big and carefully-planned projects fail, and also seen some comparatively small projects go on to achieve global success. In most cases to be honest I’ve usually had a pretty good idea of how things were going to pan out. I also remember constantly badgering product managers to implement multimedia features that are now the norm in Symbian devices: Standard earphone jacks and good quality photo cameras are two features that spring to mind – which I had been banging on about for some considerable time before they finally became commonplace in mobile phones .

When you actually care about the products that get produced – rather than the whole exercise simply being a “day-job” to fund the other half of your life – the required feature set and performance characteristics of a desireable device seem perfectly obvious, even when considering the development and support implications, and the final product BOM costings.

It always comes as a bit of a shock to discover that for many people working in the industry today, such information is neither blindingly obvious nor common sense. A lot of them – I suspect – don’t care. Over the years I’ve some to realise that people who have a genuine “feel” for this sort of thing are surprisingly few and far between, and when they go off to start new adventures, rather a big hole is left in their wake.

So it’s with great surprise and sadness that I read this week that Symbian’s David Wood – my friend and mentor from the early days of pre-Symbian Psion – has decided that it’s time to apply his well-honed analytical skills to somewhat more global concerns than mobile phones. You’ll have to read his blog to get the full story.

I know David will have a lot to contribute in his new interests, since – to me at least – he has always embodied the best attributes of a keen inventor: A rich knowledge of mathematics and physics, a scientific yet imaginative approach to problem-solving, and a great love of people whilst simultaneously appearing curiously awkward amongst them. He will be much missed.

I doubt that there’ll be a replacement for David – but then again, perhaps now there doesn’t need to be?

Having set much of the themes for the future, it’s now up to us – The Symbian Communtity – to build upon them, and bring them to life in our next-generation of Symbian devices.

I wish David all the best in his future adventures…

Video package owner-to-be

Hello World! My name is James Nash and I am (nearly) one of the package owners from the multimedia tech domain. As Martin wrote in his “Broadening the Community” post, us multimedia package owner folk will be introducing ourselves on this blog over the coming weeks. I thought I’d get the ball rolling by being the first, hopefully the others will take the plunge soon after!

I am a software engineer at Nokia, based in our London office. Currently I work in the team that is responsible for the low-level video functionality. The specifics of this are probably a bit too involved for this introductory post, so I’ll try to explain how we fit into the bigger picture in future posts. For now let’s just say we’re the guys that make sure your Symbian phone knows how to play and record videos.

At the moment there is a single “Multimedia package” in the Symbian Foundation that covers all the low-level frameworks and functionality for things like audio, video, camera and imaging. There’s quite a lot of stuff in that package so there’s a plan to split off the audio, video, camera and imaging-specific bits into packages of their own. The current multimedia package will continue to exist too but will only contain the bits that are generic or don’t neatly fit into one of the new packages. Each of the new packages will have its own package owner and I will be the one for video. (Hence my introduction as a “package owner-to-be”)

When the split happens I intend to hit the ground running which is why I’m already starting to get involved in package owner activities now. I am therefore assisting the current multimedia package owner, Jeremy Murray-Wakefield (who I’m sure will introduce himself here too) at the moment (though with a bias towards video-related things). Both of us will be attending SEE 2009 where we’ll be running a Multimedia Birds of a Feather meeting. We’ll be posting more details in due course, but we’d love it if you could swing by to chat! The more the merrier!

I’m really looking forward to working with the broader Symbian Foundation community. If you’re interested in using, developing or adding to the video package please get in touch! You can reach me at: james.nash [AT] nokia [DOT] com, via the mm package mailing list and also find me lurking in the developer forums under the username cirrus.