The OS, called Qubes, is based on Xen, X and Linux and is in a basic, alpha stage right now. Qubes relies on virtualization to separate applications running on the OS and also places many of the system-level components in sandboxes to prevent them from affecting each other.

Qubes implements Security by Isolation approach. To do this, Qubes utilizes virtualization technology, to be able to isolate various programs from each other, and even sandbox many system-level components, like networking or storage subsystem, so that their compromise don’t affect the integrity of the rest of the system.

Qubes lets the user define many security domains implemented as lightweight Virtual Machines (VMs), or “AppVMs”. E.g. user can have “personal”, “work”, “shopping”, “bank”, and “random” AppVMs and can use the applications from within those VMs just like if they were executing on the local machine, but at the same time they are well isolated from each other. Qubes supports secure copy-and-paste and file sharing between the AppVMs, of course.

The concepts of isolation and sandboxing have been around for decades, and are used in a number of applications, including hardened operating systems and some security products. And many security experts say that sandboxing is one of the more effective ways of preventing malicious code from affecting entire systems, rather than just one vulnerable application.

In a guest column in January on Threatpost, security researcher Dino Dai Zovi said that he expected more and more vendors to implement sandboxing and isolation in the coming year.

“The desktop analogue to the network firewall is the privilege separated and sandboxed application.  These mechanisms finally move the bull (untrusted data) from the china shop (your data) to the outside where it belongs (a sandbox).  While it doesn’t quite reduce the attack surface, it significantly raises the bar for an attacker through defense-in-depth.  If an attacker is able to exploit a vulnerability and execute code, they must then exploit another vulnerability in the sandboxing mechanism in order to break free and even read the user’s data,” he wrote.

Rutkowska said that she plans to release the full version of Qubes by the end of 2010, and that she may create some commercial extensions to the OS in the future.

www.threatpost.com

Word of the open sourcing first arrived from French blogger Olivier Faurax, and it’s now been confirmed by Skype itself.

In a back-and-forth to Skype Customer Support about the possibility of a Skype-supporting RPM package manager for the Mandriva distro becoming available, Faurax was told: “We are happy to be able to inform you that Skype will from now on be part of the open source community.”

When Faurax asked for clarification of that assertion, customer-support minion “Joerg” affirmed the statement, saying: “Yes, indeed, the Linux Skype version will become open source in the nearest future.”

When The Reg asked Skype about this report, a spokeswoman confirmed that open-sourcing is in the cards. “We realize the potential of the open source community,” she wrote in an email, “and believe that making Skype for Linux an open source application will help to speed up its development and enhance its compatibility with various Linux distributives.”

However, she wouldn’t provide a more-specific definition of “the nearest future,” saying only: “While it is our goal is to make Skype for Linux source code available to the community in the nearest future, we are not at a point to disclose exact release date yet.”

www.theregister.co.uk

Currently, Android apps run in the Dalvik virtual machine, but some developers may find this somewhat limited. With the new native development kit (NDK), content creators will have a few more option to create mobile programs.

The NDK could help some developers make better use of the data processing and signal processing, Google said. It also provides a set of tools used to generate native code libraries from C and C++ sources, as well as a set of headers and libraries that will be compatible with all future versions of Android.

Google warns that the NDK won’t be relevant for all developers, as it said there are “numerous” drawbacks.

“Your application will be more complicated, have reduced compatibility, have no access to framework APIs, and be harder to debug,” Google wrote on its Android developers’ blog. “That said, some applications that have self-contained, CPU-intensive operations that don’t allocate much memory may still benefit from increased performance and the ability to reuse existing code.”

The move shows that Google is making an increased effort to attract developers to its mobile, Linux-based platform. The search giant is also trying to attract content creators by making its vetting process for the Android Market more open than some of the competitors.

Apple’s App Store for the iPhone and iPod Touch is unquestionably at the top of the mobile app market, as it has over 50,000 programs and over 1 billion downloads. Additionally, developers are drawn to the potential audience of the App Store, which can be accessed by more than 40 million users.

www.informationweek.com

It’s difficult to write storage articles at this time and not focus on the upcoming 2.6.30 kernel. Why? This kernel is loaded with a number of new file systems — some of which we’ve already covered, like ext4 and btrfs. Another of the hot new file systems that is in 2.6.30 is NILFS. This file system is definitely one that you should be testing.

NILFS2 (New Implementation of a Log-Structured File System Version 2) is a very promising new log-structured file system that has continuous snapshots and versioning of the entire file system. This means that you can recover files that were deleted or unintentionally modified as well as perform backups at any time from a snapshot without a performance penalty normally associated with creating snapshots. In addition, there is evidence that NILFS has extremely good performance on SSD drives.

Log-Structured File System?

Log-Structured File Systems are a bit different than other file systems with both good points and bad points. Rather than write to a tree structure such as a b-tree or an h-tree, either with or without a journal, a log-structured file system writes all data and metadata sequentially in a continuous stream that is called a log (actually it is a circular log).

The concept was developed by John Ousterhout of TCL fame and Fred Douglis. The motivation behind log-structured file systems is that typical file systems lay out data based on spatial locality for rotating media (hard drives). But rotating media tends to have slow seek times limiting write performance. In addition, it was presumed that most IO would become write dominated (this observation is supported by a study that was summarized in a recent article). So a log-structured file system takes a new approach and treats the file system as a circular log and writes sequentially to the “head” of the log (the beginning) never over writing the existing log. This means that seeks are kept to a minimum because everything is sequential, improving write performance.

A log-structured file system, because of its design, makes it very easy to create snapshots (in NILFS they are called checkpoints) of both the data and metadata. NILFS can then mount these checkpoints (or snapshots) along side the primary NILFS file system. From these checkpoints, you can recover erased files (if the checkpoint has a date and time prior to when the file was erased) or you can use it for backups or even disaster recovery images.

Another benefit of log-structured file systems is that recovering from a crash is easier than the more typical tree based file systems (e.g. ext2, ext3, etc.). After a log-structured file system crashes, when it is remounted it can reconstruct its state from the last consistent point in the log. It starts at the head of the circular log and backs up until the file system is consistent. This point should be very close to the head so little if any data or metadata will be lost. This process is extremely fast regardless of the size of the file system.

This bears repeating – a log-structured file system recovers from a crash extremely fast and the amount of time is independent of the size of the file system. In contrast, other file systems have to replay their journal and possibly even walk their data structures to make sure the file system is consistent (i.e. run “fsck”). Everyone who has run fsck on a very large file system knows how much time it can take.

One problematic aspect of log-structured file systems is that they need to include a fairly sophisticated capability of “garbage collection” to reclaim free space. Free space needs to be reclaimed from the tail of the log, primarily the old check points, so that the file system doesn’t become full when the head of the log wraps around to the tail. There are many techniques for reclaiming space, one is covered in the Wikipedia article about log-structured file systems. The garbage collection process reclaims space from the check points (snap shots) otherwise the file system would fill far too quickly.

A Log Structured File System for Linux – NILFS

The Nippon Telephone and Telegraph (NTT) CyberSpace Laboratories has been developing NILFS (also referred to as NILFS2 since it is the version 2 of the file system) for Linux. It is released under the GPL 2.0 license and is included in the 2.6.30 kernel. It spent a great deal of time in the -mm kernels and under went much testing since it’s initial announcement.

One of the most noticeable features of NILFS is that it can “continuously and automatically save instantaneous states of the file system without interrupting service”. NILFS refers to these as checkpoints. In contrast, other file systems such as ZFS, can provide snapshots but they have to suspend operation to perform the snapshot operation. NILFS doesn’t have to do this. The snapshots (checkpoints) are part of the file system design itself.

One of the really cool features of NILFS is that these checkpoints can actually be mounted along side the primary file system. This has many, many uses, one of which is to mount a checkpoint to recover files that were unintentionally erased.

In addition to being able to recover recently erased files and extremely fast crash recovery times, there are a number of other features of NILFS that are very attractive:

• The file size and inode numbers are stored as 64-bit fields
• File sizes of up to 8 EiB (Exbibyte – approximately an Exabyte)
• Block sizes that are smaller than a page size (i.e. 1KB-2KB). This can potentially make NILFS much faster for small files than other file systems.
• File and inode blocks use a B-tree (the use of B-trees in a log-structured file system stems from the implementation which use something called segments)
• NILFS uses 32-bit checksums (CRC32) on data and metadata for integrity assurance
• Correctly ordered data and meta-data writes
• Redundant superblock
• Read-ahead for meta data files as well as data files (helps read performance)
• Continuous check pointing which can be used for snapshots. These can be used for backups or they can even be used for recovering files.

Checkpoints and Snapshots

One of the features that users can really enjoy with NILFS is the ability to recover erased or modified files. NILFS creates a checkpoint “every few seconds or per synchronous write basis (unless there is no change).” (from the kernel documentation). Then the user can select a checkpoint and convert it into a snapshot. These snapshots are preserved until they are converted back into checkpoints. Checkpoints are not preserved for the life of the file system and after a period of time the garbage collection process will recover the space in the checkpoint.

This means that users can’t recover files from a long time in the past. But there is no limit to the number of snapshots that can be created – at least until the file system volume becomes full. There are many uses for the snapshots including recovery of erased or modified files or they can be used by administrators for backups.

There are a few user-space commands that help with check points and snapshots. From the NILFS web site is an explanation of the process and is paraphrased here. The first step is to list the check points using the lscp command.

$ lscp
       CNO        DATE     TIME  MODE  SKT   NBLKINC       ICNT
         1  2008-05-08 14:45:49  cp     -         11          3
         2  2008-05-08 14:50:22  cp     -     200523         81
         3  2008-05-08 20:40:34  cp     -        136         61
         4  2008-05-08 20:41:20  cp     -     187666       1604
         5  2008-05-08 20:41:42  cp     -         51       1634
         6  2008-05-08 20:42:00  cp     -         37       1653
         7  2008-05-08 20:42:42  cp     -     272146       2116
         8  2008-05-08 20:43:13  cp     -     264649       2117
         9  2008-05-08 20:43:44  cp     -     285848       2117
        10  2008-05-08 20:44:16  cp     -     139876       7357

Notice that the output of lscp lists the date and time of the check points. Under the column labeled “MODE” is either a “cp”, that stands for “check point”, or “ss” that stands for “snap shot.” If a user does not want to wait for a check point and wants to create one immediately, the mkcp command could be used. In general you need to tell mkcp the device containing a NILFS file system otherwise it searches /proc/mounts for NILFS file systems.

To take a check point and create a snap shot, one uses the mkcp command again. In this case, one uses the command mkcp -s to create the snapshot from an existing checkpoint. You can also use the chcp command that changes a check point into a snap shot or vice versa. Again, from the NILFS website is an example of creating a snapshot.

$ sudo chcp ss 2
$ lscp
       CNO        DATE     TIME  MODE  SKT   NBLKINC       ICNT
         1  2008-05-08 14:45:49  cp     -         11          3
         2  2008-05-08 14:50:22  ss     -     200523         81
         3  2008-05-08 20:40:34  cp     -        136         61
         4  2008-05-08 20:41:20  cp     -     187666       1604
         5  2008-05-08 20:41:42  cp     -         51       1634
         6  2008-05-08 20:42:00  cp     -         37       1653
         7  2008-05-08 20:42:42  cp     -     272146       2116
         8  2008-05-08 20:43:13  cp     -     264649       2117
         9  2008-05-08 20:43:44  cp     -     285848       2117
        10  2008-05-08 20:44:16  cp     -     139876       7357
        11  2008-05-08 21:05:23  cp     -         10       7357

Notice that the chcp command changes the second check point into a snap shot. This is indicated under the “MODE” column where the second check point is listed as “ss” or snap shot. Now that the check point is a snap shot, it won’t be deleted during the garbage collection. However, you can remove the snap shot by using the rmcp command.

NILFS implements garbage collection in a unique way. It uses a user-space daemon to perform the garbage collection. This daemon is activated when the file system is mounted via the “mount” command. This also means that garbage collection can be activated at any time (if the file system is mounted).

Don’t forget that NILFS will delete check points after a certain period of time unless the check point is converted to a snap shot. The amount of time when the check point is held before being deleted is controlled by parameters in the /etc/nilfs_cleanerd.conf file. You can adjust the garbage collection (GC) parameters in the file and restart the GC daemon so that the new parameter values are used (or unmouning and remounting the file system).

You must have root access or at least sudo ability to mount a snap shot. Also recall that snap shots are mounted as read-only. From the NILFS web site example, one can mount the snapshot previously created (it was created from the second check point).

# mount -t nilfs2 -r -o cp=2 /dev/sdb1 /nilfs-cp
# df -t nilfs2
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sdb1             71679996   3203068  64888832   5% /nilfs
/dev/sdb1             71679996   3203068  64888832   5% /nilfs-cp
# mount -t nilfs2
/dev/sdb1 on /nilfs type nilfs2 (rw,gcpid=13296)
/dev/sdb1 on /nilfs-cp type nilfs2 (ro,cp=2)

The snap shot is mounted on /nilfs-cp in a read-only mode (ro). Depending upon the options and permissions on the /nilfs-cp mount point, users could copy files from the snap shot. Alternatively, the root user could restore the file(s)s for the user. Also, the snap shot could be easily used for creating back-ups. An administrator could also use the snap shot for creating a disaster recovery image of the file system. Just as a reminder the mounted snap shot, while being a NILFS file system, is mounted read-only so check points of the snapshot are not created. After you are finished with the snap shot don’t forget to unmount it and either delete the snap shot or convert it back to a check point and allow garbage collection to recover the space.

Speed, Glorious Speed

Recall that one reason log-structured file systems were developed was to increase write performance (assuming the read performance would be dominated by caching effects). And who doesn’t like increased write performance?

One of the earliest reviews of NILFS was in 2007 by Chris Samuel. He did a very comprehensive review of Emerging File Systems(how prescient was that review?). He did a very nice review of a number of file systems including NILFS including running benchmarks. The performance was good for such a young file system but even at that time it had the best performance by far for Sequential Deletes. It was even better than ZFS/OpenSolaris for most tests performed.

In Feb. 2008, there was a presentation by Dongjun Shin from Samsung as part of the Linux Storage & File System Workshop 2008 (LSF ‘08). He benchmarked NILFS, Btrfs, Ext2, Ext3, Ext4, ReiserFS, and XFS when running on an SSD device. Granted that the testing is a little old, but the results are very, very exciting. The benchmark, Postmark, simulates an email server. Two groups of files sizes were tested, (1) 9 – 15KB (S), and (2) 0.1 – 3MB (L). For each group, two tests were run with a small number of files (S), and a larger number of files (L). Figure 1 and 2 below are the test results.

Postmark Results for Small File Size

Postmark Results for Large File Size

Notice that in both cases, the performance of NILFS exceeds that of other file systems. For small files NILFS was about 25-38% faster than the nearest competitor (btrfs). For large files NILFS was about 15-25% faster than the nearest competitor (reiserfs and/or ext4).

It is pretty amazing to see such a boost in performance from a change in file system, but it does show you that the coupling of file system design with hardware, in this case SSD’s, can produce a big boost in performance. But “There Ain’t No Such Thing As A Free Lunch” TANSTAAFL. There are some current issues with NILFS and SSD’s.

There was a recent posting to the NILFS mailing list about using NILFS as the root drive for a Linux system. It was pointed out that the root file system produces a great deal of traffic. Coupled with this is the fact that NILFS file system activity can be reasonably write heavy and you have the potential for quickly wearing out SSD drives (remember that NAND chips which make up SSD’s have a limited number of rewrites). But the developers of NILFS are aware of this and a better garbage collection (GC) algorithm is under investigation.

There was also a question on the Linux kernel mailing list about the effect of age on the performance (i.e. would the performance of NILFS still remain far above others on the Postmark test after it was used for a few months?). The answer is that the developers don’t believe the performance suffers after it is used for a period of time, but there isn’t any data to back up that claim at the present time. However, virtually all files systems suffer degrading performance with age.

NILFS – It’s Definitely Worth Testing

NILFS has a great deal going for it in many regards. It is a modern file system in almost every respect (OK, no built-in RAID, but that can be worked around). The log-structured design of NILFS means that its write performance should be very, very good and there is evidence of this from the performance report that benchmarked Postmark.

Additionally, the fact that NILFS continuously creates checkpoints that can be used to create snapshots, is of great benefit. These checkpoints can be used to recover erased or modified user files. They can also be used for backups or creating disaster recover images of data. More over, creating these checkpoints or snapshots do not result in decreased performance as they do for file systems such as ZFS.

NILFS holds great promise for Linux. There are many scenarios where it would work extremely well. In particular it works very well for user directories or work directories. In the HPC world it would work extremely well for high-speed storage that are dominated by write performance. Coupling the performance boosts with the snapshot features make NILFS a potential system administrators dream file system. It is well worth trying NILFS on your system.

www.linux-mag.com

What makes oFono different?

oFono aims to provide an easy to use interface that can be used to create rich telephony applications without requiring one to have a deep understanding of the underlying technology and its limitations. To accomplish this, oFono provides language-independent D-Bus API that is extremely easy to use without generated bindings or other ‘messy’ steps typically required to use an IPC based API.

The API follows the following four basic principles:

• Consistent
• Minimal
• Easy to use
• Complete

Consistent

The API should be consistent. Functionality in oFono is broken up into Interfaces, each with well-defined set of Properties. Each interface has a GetProperties() and a SetProperty() method which is used to accomplish majority of the tasks. Naming conventions are also strictly enforced. This means that once you’re comfortable using one Interface, you should be able to easily orient yourself to use others.

Minimal

The API should be minimal; there shouldn’t be more than one way of accomplishing a task as this leads to confusion and API bloat. We also aim to maintain API compatibility whenever possible, from the very beginning. This means that oFono APIs wil generally focus on the most important use-cases. Other features will be carefully considered before being included in the APIs.

Complete

At the same time, the APIs must be rich and complete enough to enable the creation of feature complete mobile phones. Thus, APIs for all aspects of telephony are planned, incudling but not limited to Voice calls, SMS, Cell Broadcast, Supplementary Services and SIM Phonebooks.

Easy to use

oFono aims to make writing telephony applications as easy as possible. Thus oFono will take care of the core logic, including things necessary for standards compliance. This will allow application writers to focus on what they should be doing: presentation aspects and funky animated UIs.

www.ofono.org

Landing ahead of the US for “activity” in Red Hat’s Open Source Index this year were these countries, in the following order: France, Spain, Germany, Australia, Finland, the UK, Norway, and Estonia. Also among the 75 countries surveyed by Red Hat and the Georgia Institute of Technology, Denmark took tenth place.

At the opposite extreme of the open source spectrum, the study found these ten countries to be the least active, in descending order: Algeria, the Philippines, Morocco, Cameroon, Yemen, Latvia, Mauritius, Nigeria, Kenya, and Moldova.

Red Hat’s activity index measures the amount of open source now present in a country based partly on existing open source and open standards policies, along with numbers of users and producers of open source software. The annual survey also ranks countries on environmental factors that “may further, or coexist with, open source activity,” including a high number of Internet users.

But a country’s index scores for activity and environment can fall quite far apart from each other indeed. In spite of its rather lowly ninth place ranking for open source activity, the US was bested this year only by Sweden for open source environment, for example.

Although the small Eastern European nation of Moldova ended dead last for activity, it ranked 34th for environment. And regardless of its status as a software development outsourcing hotbed, India took only 23rd place for activity and 53rd place for environment. Yet Ireland, another big outsourcer, rated 19th for both activity and environment.

The survey results also broke out the two main scores into separate categories for the six areas of government, industry, and community education activity; and government, industry, and community education environment.

France, the most active country overall, also earned first place for government activity. But on community education, France ranked third, and on industry activity, only 25th.

The US took second place on community education activity and 13th on industry activity, doing better than France on both those measures. But on government activity, the US — with its ranking of 28 — fared far worse than France and a lot of other nations.

Yet the intent behind the survey is to spur global collaboration, not competition. “Red Hat hopes the Open Source Index will serve as a resource for those within the open source community along with others who are curious about open source to start building relationships and further foster worldwide open source growth,” said Red Hat VP for Corporate Affairs Tom Rabon, in a statement earlier this week.

Red Hat’s annual Open Source Index uses a framework developed by researchers at Georgia Tech.

Also this week, Red Hat announced plans to hold a virtual event called the Open Source Cloud Computing Forum on July 22.

www.betanews.com

If you’re a Linux user, though, you might now have better success than other operating systems’ users, thanks to new, block I/O data integrity infrastructure code from Oracle that’s become part of the mainline kernel.

The code is in the currently shipping Linux 2.6.27 kernel, which debuted in early October. And as a result, its developers say it could dramatically improve protections against corruption during saving.

“When an application is reading or writing against storage, one of the things that could happen is silent data corruption — ‘silent’ meaning that the application reports that data as being correctly written, but it’s not,” Monica Kumar, Oracle’s senior director for Linux, virtualization and open source product marketing

“Silent data corruption can happen more frequently than you think, and it could cause millions of dollar of damage if it’s in a mission-critical business,” she said.

Kumar also said the enhancement also means that Linux now has an advantage that no other operating system currently offers — a benefit that could serve the open source OS well as its distributions continue to compete with established proprietary systems.

“Because Linux is now the first operating system that has the T10 standard implemented, I think it does have a lead in that sense,” she said. “I’m not saying other operating systems will not follow, but for now, Linux is leading in this space.”

Building on a standard

The new block I/O data integrity infrastructure code is an implementation of an industry standard, the T10 Protection Information Model designed for end-to-end data integrity for enterprise storage systems.

The code also adds what are known as Data Integrity Extensions, a technology co-developed with storage vendor Emulex that enables integrity information to be transmitted from memory.

Kumar explained that the data integrity feature works as an additional checkpoint at the operating system level, verifying the data being written or read. As a result, the process is designed to make data corruption less likely.

“The validity of I/O operations is assured through the exchange of verification information during data transmission,” she said. “The key is creating integrity metadata, so as the data travels the data path, the integrity metadata is validated.”

According to Kumar, Oracle and Emulex were working on the data integrity feature since mid-2006. In July of this year, the code landed in the Linux kernel development tree, ultimately becoming included in the final 2.6.27 kernel release in October.

Currently, the technology works with the Linux-based Ext3 filesystem, though Kumar noted that work is ongoing to validate the data integrity technology against other filesystems as well.

At the same time, Oracle and Emulex are pushing an early-adopter program to get a small group of end users testing and deploying the technology, which could help demonstrate its real-world potential.

Wider benefits

Since the data integrity technology is now part of the mainline Linux kernel, it is available to all Linux kernel users — and it could end up benefiting others beyond just Oracle and Emulex. Among two of the likely beneficiaries are Linux distribution vendors Novell and Red Hat.

“It goes without saying that any technology that improves data integrity is valued by Red Hat Enterprise Linux customers,” Nick Carr, marketing director at Red Hat, told InternetNews.com.

Michael Applebaum, senior product marketing manager at Novell, is also enthusiastic about the new data integrity technology.

“We see strong customer demand,” Applebaum told InternetNews.com. “Our heavy database users … for example, data warehousing vendors like Teradata, will benefit from enhanced data integrity.”

“We think such advanced features further establish SUSE Linux Enterprise Server as the operating system of choice for mission-critical data center workloads such as major database, middleware and line of business applications, backed by our strategic partners SAP and Microsoft,” he added.

For the present, Oracle is talking up its contribution as it and Emulex build an early adopter program to get the technology more widely deployed among storage vendors.

As a result, Kumar also suggested that storage vendors will soon be announcing gear that is specifically optimized for the technology, but provided few additional details.

www.internetnews.com