Our blog

Friday, 22 July 2011 | Disclaimer | Comments [6]
Category: MongoDB

By Ivan, PhD., a Java developer at Murano Software’s team

This is the last post from our series about MongoDB. We have shown you the features, the drawbacks and now we will show you how it handles Java.

As mentioned before, MongoDB supports a number of drivers for different platforms; however, since it's mostly designed to operate in the Web's distributed environment, it comes naturally along Java technology. But enough words said, instead let's jump straight to code and see what it really looks like from the inside. To make it even more exciting, we put in some competition flavor by comparing MongoDB on Java with something very common and widely used, something like MySQL. So, we are about to assemble some kind of roughly made comparative performance test for MongoDB and MySQL with bulk inserts, selects and updates. Before we move on, I beg you not to take this performance test too seriously, since it's in no way a complete, redundant or ultimate performance test, and it's not intended to say some heavy 'yes' or 'no' toward a specific solution. It's merely an applied example to make your journey with MongoDB interesting and useful.

There is plenty of detailed documentation for both MySQL’s and MongoDB’s installation setup processes. They both are available as a binary distribution. If you are a Linux user, I encourage you to use repository distributions. Let's assume we have our environment set up and running, and we ready to start coding.

We need some generic testing and reporting suite and service running our insert/select/update routines for available database providers. You can download complete code to look into and play with it using this link:

Let's take a closer look at MongoDB’s database service, MongoDbOpService.java:

<code>

...

public class MongoDbOpService implements DatabaseOpService, InitializingBean {
...
    @Override
    public void cleanUp() {
        col.remove(BasicDBObjectBuilder.start().get());
        col.ensureIndex(BasicDBObjectBuilder.start().add("field1", 1).get());
    }
  @Override
   public void afterPropertiesSet() throws Exception {
        Mongo mongo = new Mongo(host, port);
        col = mongo.getDB("test").getCollection("test");
        cleanUp();
  }
  @Override
  public void select(int numOfOps) throws DatabaseOperationException {
       for (int i = 0; i < numOfOps; i++){
       col.find(BasicDBObjectBuilder.start()
      .add("field1", new BasicDBObject("$gt", i)).get());
  }
  }
  @Override
  public void update(int numOfOps) throws DatabaseOperationException {
       for (int i = 0; i < numOfOps; i++) {
      col.update(new BasicDBObject("field1", new BasicDBObject("$gt", i)),
      new BasicDBObject("$set", new BasicDBObject("field2", 1)), false, true);
  }
  }
  @Override
  public void insert(int numOfOps) throws DatabaseOperationException {
      for (int i = 0; i < numOfOps; i++) {
          BasicDBObjectBuilder builder = new BasicDBObjectBuilder();
          builder
             .add("field1", (int) (Math.random() * numOfOps))
             .add("field2", (int) (Math.random() * numOfOps))
             .add("field3", (int) (Math.random() * numOfOps));
         col.save(builder.get());
        }
  }
...
}

</code>

We start by just creating Mongo instance, with its constructor accepting the host address as a parameter. Then we want to create a test database and collect in it. The trick is that we don't have to worry about whether the database with the given name exists. getDB() will return existing names or create a new one for us, and the same applies to getCollection(). Since we are going to run through our routines several times, we want to ensure we have a clean collection for each pass, and we can do it by calling the cleanUp() service method. It will remove all documents from the collection in case this is not the first run and ensure (create) index on the field ('field1') that we are going to query later on. If you did the same thing from a JavaScript console, it would look something like this:

<code>

> db.test.remove({})
> db.test.ensureIndex({“field1”:1})

</code>

Where 'test' is the name of the collection, remove is given an empty JSON object as a filter argument, and ensureIndex is given an object with only one name as an index target field, while '1' indicates ascending indexing order. Worth mentioning is that almost everything inside MongoDB uses json extensively for filtering, querying, update operations, etc. Also note that there is nothing like data schema for the created MongoDB collection.

The next point of interest is the insert(int N) call that will insert N simple objects into our test collection, which in json will look like this:

<code>

{
     “field1” : random_int,
     “field2” : random_int,
     “field3” : random_int
}

</code>

Now, when we have N documents in our collection, we can start playing and selecting them. The select(int N) call will make N queries to test the collection, trying to fetch all documents that have their field2 value greater than some integer (0,N]. Note that such query returns the cursor, so it's usually a good idea to consider paging and result limiting in your live application, especially on large data results. Otherwise you will definitely hit a cursor timeout issue or some other nasty distributed environment issues. The very same query in JS console would look like this:

<code>

> db.test.find({“field1”:{$gt:i}})

</code>

Note how we use the filter operator keyword '$gt' (greater than). There are two things to note here. First, there are plenty of such keywords in mongodb query syntax used for filtering, updating and some other cool stuff. You can find all the details in manuals. What is more important to note is that we actually use json object as a value for “field1.” So your documents can contain any number of nested documents in field values, as well as arrays and functions. This can lead to very interesting applications, including stored procedures. Just take a look at this example object:

<code>

{
     _id : ObjectID(“4e0a078e69fea677c91d3742”),
     name: “John”,
     address:{
                             city:“Moscow”,
                             street:{
                                                     name: ”Lenin”,
                                                     type: “Square”
                                                 },
                             zip: 12345
                           },
     sayHi : function(){print('Hello from function!');},
     some_list:[1245968390, 2859408375, 8756203941],
     some_object_list:[
                                            {field1:”something1”,field2:1},
                                            {field1:”something2”,field2:-1}
                                         ]
}

</code>

Even though it can look confusing, it turns out to be a very convenient way of making things work. Also, since this is actually an object, you can use “dot notation” to access any nested field. So if we match the object from the last example, we can do the following:

<code>

> a = db.test.findOne({name:”John”})
> a.address.street.type
Square
> a.sayHi()
Hello from function!

> a.some_list[0]
1245968390

</code>

Finally, update(int N) call makes N update operations, trying to find objects with the same query select() does, but this time we want to change 'field2' value of matched objects. Note that we are using the '$set' atomic modifier here, so that this update won't lock the object being changed, only the field value is to be updated.

Here are the results I've got on my local machine. I have to mention that even the simple and quick-made test we used shows how ridiculously slow bulk operation can be with the traditional relational approach. I basically stopped on the 10k set because it takes way too much time with little difference in results. In the meantime, using MongoDB, I've had times when I generated millions of documents for the demo collection, and it took a matter of minutes.

Test: MongoDB vs MySQL quick performance test (insert/select/update)
Environment: Ubuntu 11.04 Linux x86_64 2.6.38, i5-2400, RAM 8Gb

Service OpNum Ins InsAvg Sel SelAvg Upd UpdAvg
MongoDb 10 1 0.1 1 0.1 1 0.1
MySQL 10 533 53.3 3 0.3 599 59.9
MongoDb 100 19 0.19 0 0.0 8 0.08
MySQL 100 5145 51.45 42 0.42 5866 58.66
MongoDb 1000 79 0.079 5 0.0050 53 0.053
MySQL 1000 49695 49.695 966 0.966 52245 52.245
MongoDb 10000 435 0.0435 18 0.0018 67182 6.7182
MySQL 10000 490376 49.0376 40992 4.0992 863177 86.3177

clip_image002

                                   Figure 2 - Average select time, ms/record

clip_image002[5]

                                    Figure 3 - Average update time, ms/record

clip_image002[9]

Epilogue

After having a brief look at MongoDB's features and trade offs, one can ask oneself, is this piece of technology production ready, and could I use it in my project? Well, it's up to you, but my word is that you should give it a try. The test we were using as an example cannot be taken to prove either MongoDB or MySQL is better. It just shows how these two systems can handle one specific test. What is absolutely exciting about MongoDB is that it's really easy to start working with. It's just a matter of an hour to layout some basic configuration and client code. There are huge advantages and yet a price to pay.

Wednesday, 20 July 2011 | Disclaimer | Comments [1]
Category: MongoDB

In our previous posts we introduced you to MongoDB as well as it’s features. In this part we will be talking about the

Trade offs

Finally, it’s payback time. Of course, we are not expecting to get a scalable and high-performance database with little to no cost. Current MongoDB drawbacks should be divided into two groups. The first group is caused by fundamental differences in approach and the price we have to pay – consistency. This means that, as soon as MongoDB is your weapon of choice, your application logic has to provide workarounds for concurrency issues and consistency checks. It's good to know that MongoDB has two options to tackle the last problem – write with concern (WriteConcern) allows you to go around the “save now write later” issue, and it can be a powerful tool in skilled hands, while the slaveOk setting allows you to force “read from primary only” behavior, so you are no longer trapped with replication sync issues. And of course, versions and atomic updates help with concurrency.

I should name the second group “age-related problems.” These are the drawbacks that we are facing now, but they are more or less likely to go away in the future. Some of these drawback fixes are harder to implement and/or have a lower priority. To name a few:

As mentioned before, queries in MongoDB work perfectly when the corresponding index is there. But if your query filter has some not-indexed fields, you will end up with a very long query time. There is a way to hint or force MongoDB to use the existing query without that field, but you have to understand indexing mechanics and use this feature carefully. Another thing is that MongoDB keeps all indexes in memory, and even though memory size is usually quite large for servers, it’s still limited, and you have to keep that in mind and make sure you are not creating too many indexes.

Also when querying, you have to keep in mind that not all query types currently support or use indexes; therefore, using something like MapReduce, where clause or some regular expressions, you could end up with a long query time. Due to JavaScript engine limitations, MapReduce queries are processed in a single thread and can lock down the whole database instance. Eventually, indexes could be very productive for heavy-read applications and could show up as counterproductive in applications with lots of writes.

MongoDB is supposed to be operated from a secured/trusted environments and only has basic authentication implemented. You have only two alternatives for user access rights: it's either a read-only user or an admin user with all write/read privileges. The user gets access database-wide, and it's not possible to set up different access per collection. Also, the basic auth mechanism currently doesn't work for sharded configurations.

Not a big deal, usually, but still a limitation of single document size. As for version 1.8, it’s 8Mb and assumed to increase in later releases.

MongoDB returns the cursor as a query result. There is such a thing as a cursor timeout. So, especially when operating in distributed configuration (shard + replicaset), you should consider paging queries in a smaller chunks.

Last but not least, even though there are a number of MongoDB drivers for all platforms, they perform differently. Whether it's a platform or a driver implementation, it's always a good idea to try and test-driver performance in your environment before making a final decision.

Stay tuned for our last posts where we will be talking about how MongoDB and Java

Friday, 15 July 2011 | Disclaimer | Comments [5]
Category: MongoDB

By Ivan, a Java Developer at Murano software’s team.

In our previous post we have introduced you to MongoDB which has been getting quite the attention lately. We are continuing our series about Mongo, and more precisely with..

MongoDB Features

MongoDB distribution includes a JavaScript interactive shell that can be used to access, query, update, configure and administer databases on local and remote hosts. Having that JavaScript is the language of choice. All MongoDB shell operations are JS calls, and JSON is the common way to define objects here.

MongoDB is fast, I mean really fast, when it comes to search and update operations, as long as you have the corresponding index. It uses B-tree based indexes. Each collection can have more than one index, and each can include as many fields as you want (compound indexes supported).

MongoDB doesn't support transactions as most of the ACID-compliant relational databases do, but it has its own way to tackle the concurrency problem – atomic update operations on the document level. This basically means you can update some of the document's fields without affecting the rest of them. There is a kind of convention, called “update-if-current,” to make sure you're not saving an outdated document using versions, or a filtering clause for the update operation. MongoDB also supports the findAndModify operation that updates the value and immediately returns to the previous or current value. This can be useful in many cases, like queues or sequencers.

When it comes to aggregating queries, MongoDB has MapReduce support. This distributed computation technique is another gift from Google to humanity. It can look confusing to a common SQL-minded person at first glance, but a patient user should have no problems mastering it quickly. Some people like it to the extent they develop something like CoachDB, which is almost completely based on MapReduce and definitely is a worthy piece of technology.

MongoDB was born with scalability in mind, so it implements replication and sharding naturally. Replicas are organized into so-called ReplicaSets that consist of one primary host (used for writes) and secondaries (currently up to 7). This is a perfect choice for failover and redundancy. If the primary host goes down, the new one will be “elected” automatically from the working secondaries.

MongoDB provides horizontal scalability with the sharding implementation. MongoDB sharding configuration works on the collection level, evenly distributing chunks of collection data among available cluster shards using the shard key. The shard key can be one or more of the existing document fields or sometimes a specifically generated field to ensure shallower load distribution among shards. Sharded and unsharded collections feel exactly the same for the client application.

These two scaling techniques can and should be used simultaneously, as when each shard for the sharded collection consists of a ReplicaSet.

Look forward to Part 3 of our Hands on MongoDB series!

Wednesday, 13 July 2011 | Disclaimer | Comments [4]
Category: MongoDB

By Ivan, a Java developer at Murano Software’s team

«Why is it you're always
too small or too tall?»,
The Mad Hatter, Alice in Wonderland

 Prologue

Indeed, there is no such thing as an ultimate solution. Every time we face a problem, even if we are lucky enough to have a precooked solution at our fingertips, we still have to tweak, adjust and adapt it to a specific case's demands. Sometimes, the “adjustment” part can take a while, and therefore putting the right tool in the right place has a great impact on the overall project development efficiency.

From the day the Web was born, it never stopped growing in both size and quality. Whether you are developing an online shop, developing a news portal or trying to get to your customer with a company site, databases are always going to be a major building block for your solution. In terms of maturity and functional completeness, nothing can compare to relational databases. Of course, there are plenty of RDBMS, each with its own advantages and flaws, but they have proved that their strict rules and functionality can give a rock-solid basis for any web project.

However, in some cases, the relational approach can be too “tall.” Schema limitations and weak scalability support can be a decisive factor leading you to look for another solution. And object-oriented databases  are among the alternatives here. So, without further delay, let's get our hands on MongoDB – an object-oriented database that’s gaining popularity recently.

Essentials

MongoDB is an open (GNU AGPL 3.0 license), schema-less, non-SQL, document-oriented database management system developed by a 10gen. It has been developed since 2007 with the first public release in 2009. It's available for download from http://www.mongodb.org. MongoDB has a variety of drivers to work with both officially supported (Java, C#, CPP, Python, PHP, Ruby, Scala, Erlang, Perl and others) and community developed. And, of course, MongoDB is available for all Mac, Solaris, Windows and Linux users, with sources available so you basically could try building it yourself for any platform.

So what is “schema-less” all about? The deal is that MongoDB does not have any schema or any other kind of user-defined restrictions for data storage. When using a relational database, you have plenty of work to do in order to prepare your domain entities for storage in tables and rows. You define each field type, add restrictions, sometimes even put some logic in triggers and stored procedures.

This is not the case with MongoDB.  What once was a record in a relational table (usually more than one table) now becomes a document. Each document can have an unlimited number of fields. A field is a pair of key (unique string identifier) and value. A value can be of any supported type as well as an object (document) or array itself. You can use dot notation to access values at any field level. Documents are stored in collections. But since you don't have any schema, every document in a single collection can have any set of different fields. Of course, it's a good practice to put different objects in separate collections.

Obviously, this kind of unrestrained storage is hardly compatible with SQL, and there is no such thing as “joint” operations in object-oriented databases. So, let's see what MongoDB has to offer instead.

BASE vs. ACID

While most, if not all, relational databases offer their prayers to ACID (Atomicity, Consistency, Isolation, Durability), this evil god demands sacrifices. According to Eric Brewer's CAP theorem, there are at most two of three properties a shared data system can have: Consistency, Availability and Partition tolerance. And while RDBMS strive hard for Consistency and tolerance, Availability is left behind.

An object-oriented database’s logic instead declares that, in the cases where this is applicable, consistency can be given up in favor of high availability and scalability. This is sometimes referred to as a BASE acronym, which stands for Basically Available, Soft state, Eventually available. It means it's okay to go on without locks and transactions. It's okay to have data in a soft state (we can consider write operation as complete before the data is actually written to disk).The data will eventually become consistent, and in most cases, there are plenty of other ways to work around concurrency.

Now we will make a small break. Have we piqued your interest? Look out for Part 2, where we will be introducing the Features of MongoDB

Tuesday, 07 September 2010 | Disclaimer | Comments [3]

In one of our previous posts, we already described the technical details of Multidraw, our rich cloud application that showed how Windows Azure cloud computing works. The solution was generalized into an open source framework, which has been released to the Microsoft Developer Network. This significant event highlights how Murano Software is a leading expert in Windows Azure on the global arena. To see Multidraw in action and learn more about Murano’s expertise in Azure, check out our new video:

Tuesday, 25 May 2010 | Disclaimer | Comments [3]

By Pavel, a .NET Developer on Murano Software’s team

One of the new, pleasant features of Silverlight 4 RC is comprehensive printing support, enabling hardcopy reports and documents, as well as a virtual print view, independent of screen content. We used this feature while updating RCA POC project phase 2, and we want to give a little overview of how to use this feature.

Printing API

The PrintDocument class provides printing capabilities from a Silverlight application. A developer can handle three events of this class: StartPrint, EndPrint and PrintPage. The StartPrint event is used for special handling or setting up before printing begins. The EndPrint event is used for clean up or other operations after printing is complete. Use the PrintPage event to generate page content for printing by setting the PrintPageEventArgs argument.

The PrintPageEventArgs class contains three main properties: PageVisual, HasMorePages and PrintableArea. The PageVisual property sets the element for printing. It can be the layout root of Silverlight content or the name of UIElement. The PrintableArea gets the size of the printable area. You can use the HasMorePages property for printing multiple pages. If HasMorePages = true, then the PrintPage event is called again to print the next page.

Sample:

private void PrintAll_Click(object sender, RoutedEventArgs e)
{
	PrintDocument docToPrint = new PrintDocument();
	// prepare to print
	docToPrint.BeginPrint += (s, args) =>
	{
		// do something
	};
	// set UIElement
	docToPrint.PrintPage += (s, args) =>
	{
		args.PageVisual = this.StackOfStuff;
		args.HasMorePages = true;
	};
	// ending prepare to print
	docToPrint.EndPrint += (s, args) =>
	{
		// do something
	};
	// start print
	docToPrint.Print("Entire Screen Sample";);
}

Print Preview

We reviewed the main features of Printing API. Now let's look at how to implement a preview of the printed content. It’s easy. You have to create a User control where your printable data will be showed in the way you need. Then you set this control to the PageVisual property of the PrintPageEventArgs and print it.

Examples

The following code example shows a PrintPage event handler in RCA POC project, where stPage is a grid in the PrintPreviewBox control and the document is an instance of the PrintDocument class:

document.PrintPage += (sender1, args) =>
	{
		((Grid)stPage.Children[pagesPrinted]).Width = args.PrintableArea.Width;
		((Grid)stPage.Children[pagesPrinted]).Height = args.PrintableArea.Height;
		args.PageVisual = stPage.Children[pagesPrinted];
		pagesPrinted++;
		args.HasMorePages = stPage.Children.Count > pagesPrinted;
	};

Then we invoke the method Print() of document object. This will show a print dialog, where we can select a printer. After that, Silverlight will generate the document and send it to the printer. You can see a preview page from RCA POC project on figure below.

 Preview of report with comments 

Conclusion

As you can see, Silverlight 4 RC allows a developer to print documents with any content, whether photos, charts or text.

Monday, 24 May 2010 | Disclaimer | Comments [5]

By Denis, a .NET Developer on Murano Software’s team

One of the updates in the RCA phase 2 was a porting our database from Azure storage to Azure SQL. Minimal changes we needed to make were on the server (Azure web/web role) and Azure SQL. We also changed some scripts on the client to increase performance. Let’s look at the changes in the main two steps:

Changes in the Azure SQL

The main differences between Azure storage and Azure SQL are:

  1. Each table in Azure storage has two key fields: RowKey and PartitionKey.
  2. A database in Azure storage is not relational, so no joins, group by and order by.
  3. Windows Azure storage can return either 1,000 rows or 4 Kbytes.
  4. Two key fields must be properly designed for good performance.

We also want to note that Azure SQL has some limitations as compared to traditional RDBMS. We couldn’t make a connection to Azure SQL from Visual studio before version 2010 RC, and we had to create copy of the Azure SQL database on the local SQL Server, make a model and change the connection string to the database in Azure SQL. But this problem was resolved in VS 2010 RC, and we can work with Azure SQL directly.

Thus, we removed PartitionKey and RowKey, as well as TimeStamp, from the structure of each table because these keys can’t be used as primary keys in relational database. Then we formed primary keys and relations between tables. We didn’t find any tool to migrate the database from Azure storage to Azure SQL. So we implemented sql-script to create our database structure by hand.

We want to note that you can work with Azure SQL, as well as the traditional SQL Server, using SQL Server 2008 R2 Management Studio CTP. One of the most pleasant features of this tool is the script generator that can generate script from an existing database. A generated script is adapted to Azure SQL and can create a database in Azure SQL. So you can create a database in the local SQL Server and use it in your Rich Internet Application before releasing and generating the database in Azure SQL only for the production version.

Changes on the server (Web role / Azure Web)

We used the data access layer described in this blog (Porting Silverlight RIA to Windows Azure) to work with Azure storage. The next step was to decide what kind of technology we would use to work with Azure SQL. We chose the LINQ to SQL technology. So we changed the previous model to a model generated by the DBML Code Generator. Our new model has to be inherited from the IUpdatable interface, so we implemented a partial class of our model that inherits from the IUpdatable interface. A generic implementation of the IUpdatable interface for LINQ to SQL is provided under MS-PL license. You can find it here and use it in the partial class of your model.

These are all minimal changes you need to make to port your Azure storage to Azure SQL. Good luck.

Wednesday, 21 April 2010 | Disclaimer | Comments [0]

By Stanislav, a Senior .NET Developer on Murano Software’s team

The latest version of Silverlight delivers hundreds of features, such as a full set of form controls, enhanced data-binding support, printing API, UDP, webcam and microphone support, full trust in out-of-browser, and so on. But it doesn't have an out-of-box solution to stream video or audio captured from a microphone and Web camera via a network. Current API allows you to capture video/audio data from media devices, and it has a way to play back media information using MediaStreamSource and MediaElement control. Also, API has the UdpAnySourceMulticastClient class, which is a client receiver for multicast traffic from any source, also known as Any Source Multicast (ASM) or Internet Standard Multicast (ISM).

Silverlight 4 gets support for microphones and webcams. It allows us to have access to the raw streams. The added support opens a lot of opportunities for new types of applications.

During our work on http://code.msdn.microsoft.com/rca (source code is available under the MS-PL license), we created an application that allows users to communicate over the local network and to establish video/audio conferencing.

The classes that expose this functionality live in the System.Windows.Media namespace.

There are two classes that give us access to audio and video (AudioSink, VideoSink).

To obtain video information from a video input device in Silverlight, you have to derive a custom video sink from VideoSink, which exposes several virtual callbacks:

  • OnCaptureStarted
  • OnCaptureStopped
  • OnFormatChange
  • OnSamples

When you derive from VideoSink, you must provide overrides for the callbacks in order to compile.

protected override void OnSamples(long sampleTime, long frameDuration, byte[] sampleData)
{
	// Some code here...
}

The idea was to split data related to the particular frame into a set of packets, renumber them, add additional information and send them over the network.

On the client side, the application receives the mentioned packets, orders them, recovers the frame structure and passes frames to MediaElement, using MediaStreamSource.

The sources can be downloaded here. All logic related to the media chat is located in the PSO.Client.UDPMediaChat project (see Figure 1). It contains the following classes:

Class name Description
MediaFrame Contains all related data to the media sample
UdpAudioSink,
UdpVideoSink
These classes are responsible for the capturing raw media data and passing it to the appropriate media channel.
VideoPacketChannel,
AudioPacketChannel
These classes are responsible for the preparing data for the transmission. They split media samples into a set of packets and pass them to the transmitter.
NetPacketTransmitter It contains the logic of the transmission and receives NetPackets over the network.
NetAudioPacketSerializer,
NetVideoPacketSerializer
These classes are responsible for the logic of packet serialization and de-serialization.
NetVideoPacket,
NetAudioPacket
Stores all necessary media data that is prepared to send over the network.
StreamingServer Contains logic that allows users to organize raw media data transmission over the network.
MediaFrameSource Contains media buffering logic.
RawMediaStreamSource Contains the logic of the prepared media samples for the playback by MediaElement

Table 1. The description of the most important classes.

Solution structure

Figure 1. Solution explorer

Sequence diagrams show how the application processes media samples before sending them over a network.

We streamed raw data without any compression. Unfortunately, Silverlight doesn't supply codecs yet, but we suppose that it's possible to compress media streams using COM Object Access in Trusted Applications.

Figure 2 shows that we have two media data sources (VideoSink and AudioSink). They pass media data to the corresponding media packet channel, which is responsible for the media frames’ (samples’) processing and further transmission. We want to note that these sinks work separately in different threads. Also, the media channel contains frames splitting logic into a pile of packets. Each packet is sent over the network separately.

 Sequence diagram shows simplified algorithm of raw media streaming

Figure 2. Sequence diagram shows simplified algorithm of raw media streaming.

When you use a multicast client, the first thing you have to do is to join the group, using the known multicast IP address (ex: 224.0.0.1 - The All Hosts multicast group that contains all systems on the same network segment) and the port (ex: 9999). The address block 224.0.0.0/24 (224.0.0.0 to 224.0.0.255) is designated for multicasting on the local subnetwork only. When the connection has been made, you can start send and/or receive from the group.

There are several security restrictions on connecting to multicast groups in Silverlight.

The security policy checks included in the Silverlight runtime are designed to prevent networking threats like DoS attacks, DNS rebinding, reverse tunnel attack, etc. Currently, it's not possible to connect to remote ports less then 1024. Before a multicast client is allowed to join a group, the Silverlight runtime implements a protocol check to verify that the client has been granted permission to join the group and receive datagrams.

Sink produces raw media data by the callback:

protected override void OnSamples(long sampleTime, long frameDuration, byte[] sampleData)
{
	// Some code here...
}

These data are packed in a media frame and passed to the corresponding media channel. The channel splits the media frame into a set of packets because UdpAnySourceMulticastClient restricts the amount of data that can be sent at once. So the application cuts huge media frames that contain raw media data into small pieces and sends them over the network.

Sequence diagram shows simplified playback algorithm of the raw media stream

Figure 3. Sequence diagram shows simplified playback algorithm of the raw media stream.

On the client side, UdpAnySourceMulticastClient receives raw bytes. It passes them to NetPacketTransmitter. The transmitter verifies the destination address and “unpacks” the packet.

Then the transmitter notifies the media channels that the packet with media data has been received.

Each channel verifies the ability to process the incoming packet. The logic of the channels could vary drastically, depending on the nature of the received data. For instance, VideoChannel uses a special mechanism for the video frames recovery from a set of packets. When the media channel decides that a subsequent frame is received completely, it passes it to the MediaFrameSource.

This class manages media frame queues for both audio and video frames. Each received frame is added to the particular queue. The frames from the queues are pulled by the MediaStreamSource.

If the queue is empty, the thread that serves MediaStreamSource is suspended until the queue receives at least one frame.

Audio chat window

Figure 5. Audio chat window

A user can perform a call to other user by clicking on the phone icon at the left-side of the main window. The other participant has to confirm the call. If the call is confirmed, the special media control will be displayed (see Figure 5). By default PSO establishes an audio conference. If you want to perform a video call you have to press the film icon. In this case your application instance will start a video transmission and the other participant will be able to see you. The video chat window is displayed on figure 6.

Video chat window

Figure 6. Video chat window

We successfully realized a proof of concept that shows the possibility of video conference software development on the Silverlight 4 technology. Also, we built a universal extensible framework that allows users to transmit data in the local network.

Wednesday, 21 April 2010 | Disclaimer | Comments [1]

By Denis, a .NET Developer on Murano Software’s team

We are glad to present the release of the new extended version of Rich Cloud Application (RCA) that was announced in the last Rich Cloud Application blog post. The main enhancements of this version are using Silverlight 4 RC features, SQL Azure and Microsoft SyncFx:

  1. Using Silverlight’s comprehensive printing support. Any registered user can make a hard copy, as well as a virtual printing view, of any report in the system.
  2. Using a microphone and Web camera in the world of today is the standard de facto of user collaboration on the Internet. We extended the private chat with the possibility of using a microphone and a Webcam for more effective cooperation.
  3. Another cool feature of Silverlight 4 RC is multicast networking, enabling enterprises to lower the cost of streaming broadcast events. We implemented video/audio stream broadcasting for video chat in the system by using UDP multicast.
  4. Implementation of a new, complex rating system requires us to move the database from Azure storage to Azure SQL. All changes that are needed to move the application from Azure storage to Azure SQL will be described in the next blog post.
  5. We implemented the possibility of comfortably working with our system in off-line mode. The user can run the application in out-of-browser mode and work with cached data. Any changes in cache will be synchronized with storage in Azure SQL by Microsoft SyncFX while online.

Click below to take a look at screenshots of the application in action or try it online at http://pso.cloudapp.net!

The full source of code, except synchronization, is available for download at the MSDN Code Gallery under the MS-PL license. You can find some interesting solutions and work-arounds for creating peer-to-peer video chat and printing support.

So stay tuned and subscribe to our blog to get fresh news about Rich Cloud Application!