Monday, December 31, 2007
Sunday, December 30, 2007
- Firewall support - the BB is inside the corporate firewall, the iPhone can't access it. We use Juniper Network Connect which is a Java based VPN solution on MacOS/XP.
- WiFi support - we use LEAP to login to WiFi at work, need support for LEAP on iPhone, it works fine on my MacOS X laptop, should be a simple feature to add.
- Exchange support - I can't use the IMAP workaround due to firewall issues, properly integrated Exchange email and calendar support is what everyone is asking for.
- Ideally RIM will port the Blackberry application suite to the iPhone, like they did for the Treo...
Friday, December 28, 2007
Sunday, December 09, 2007
Each year, CMG considers exceptional individuals who have made significant contributions to our profession over their entire professional careers as possible recipients of the A.A. Michelson Award.The recipient is nominated and chosen by the past winners, and it has been granted to one person every year since 1974. This year I was very honored to be the recipient! It was announced on Monday 3rd December, at the start of the CMG07 conference. I gave an acceptance speech, that I have summarized into a blog post.
Albert Abraham Michelson, for whom the award is named, was known for his outstanding technical accomplishments in measuring the speed of light as well as for his role as a teacher and inspirer of others. CMG presents this lifetime achievement award to a single individual to recognize and encourage the same combination of technical excellence and professional contributions found in only an exceptional few.
Thanks to everyone who has supported me over the years, reading the blogs, papers and books; coming to training classes; the engineers at Sun who fixed my bugs and added the features I lobbied for to make Solaris more measurable; and the friends, mentors and mentees that I have worked with for many years.
Saturday, December 01, 2007
Development is very rapid, agile and iterative. Features go through rigorous statistical A/B testing, and have to show a significant benefit before all the users get to see them. There is a great deal of freedom to try out ideas and an extremely analytical approach to picking the winners.
Wednesday, November 21, 2007
Ruud is an advocate and expert in OpenMP, and has just published an excellent book on the subject. "Using OpenMP". Published by MIT Press. The work was done together with Barbara Chapman and Gabriele Jost. The ISBN numbers are ISBN-10: 0-262-53302-2 and ISBN-13: 978-0-262-53302-7.
Even if you aren't using OpenMP, the concepts for performance and scaling are the same for everyone...
Friday, November 16, 2007
CMG moves around the US, last year it was in Reno NV, before that in Orlando FL.
On Sunday 2nd December there is a workshop day. I'm co-presenting (with Mario Jauvin) a half day tutorial on Capacity Planning and Performance Monitoring with Free Tools (for the third year). We updated our slides a bit more this year to track the latest versions of the free tools.
During the week I'm giving a new paper on Millicomputing, that introduces the concepts I have been discussing on my Millicomputing blog, and talks about some of the management challenges of managing large numbers of small computers.
I'm also giving a half day training class on Unix/Linux Performance Management, which is based on the materials I developed over many years in my books and talks. It is still somewhat biased towards Solaris since I know that better, but I have been adding more on Linux and AIX as that is the world I currently live in at work (Netflix runs RedHat Linux and AIX on IBM P-series) and in my Millicomputing sideline (I run Ubuntu on my development host and linux 2.6 on my gumstix).
The conference contains a wide range of content, there is:
- a large vendor exhibition showing the latest tools, vendor presentations and user group meetings to talk about their next steps.
- everything from beginner level introductions to deep mathematical theory.
- coverage of networking, storage, CPUs, application level performance.
- discussion of Linux, Solaris, AIX, Windows, Mainframe and all combinations
- panel discussions on virtualization, service oriented architectures, scaling large sites
Tuesday, November 13, 2007
The SDK is 55MB, there is available Eclipse support.
Sample applications include Google Maps, mail, webkit based browser and the usual Phone and PIM stuff. The user interface seems to be based on a new set of Java classes, but there is also direct access to the display in C/C++ using SGL for 2D and OpenGL ES for 3D, and audio and video players for things like mp3 and H.264.
From a first pass through the docs, a lot of work has gone into this already. It looks as if it should be portable
to a homebrew phone without much difficulty, and from then on, we are likely to be leveraging a much larger community than the other open alternatives. It appears that the current state of the OpenMoko project is that they have made some progress and have a somewhat usable toolkit written in Python, but that C/C++ development based on GTK+ is painful, and documentation is scarce. It looks to me as if Android has taken the same basic inspiration as OpenMoko, but with a much more professional execution, and with a Java based developer platform.
We will be discussing this at the Homebrew Mobile Phone club meeting on Wednesday. We are holding the meeting at Mozilla's offices and will also be discussing their work on a mobile version of the Mozilla browser.
Sunday, November 04, 2007
Wednesday, October 17, 2007
Thursday, October 04, 2007
I have *no inside knowledge* of anything going on at Apple, this is all conjecture, OK?
Product teams have four big things that can be traded off against each other to some extent.
- The release date
- The available resources
- The feature set at launch
- The product quality at launch
- US Market Only - no internationalization
- Basic camera support, no video capture (not even demo'd at the launch event)
- Relatively few preferences and options
Wednesday, October 03, 2007
I haven't hacked my iPhone, and it upgraded to 1.1.1 cleanly. The new features and upgrades are welcome, but the upgrade process didn't clear out the ever increasing amount of "Other Memory". I had resorted to removing videos from my iPhone to make space and avoid out of memory errors. You can see 1.6GB in the display below.
I went through the settings, cleared out deleted mail and safari history and it made no difference. I deleted all the content and it made no difference, so I re-loaded the iPhone from scratch, and it dropped to 14MB after I had put my content back.
There are some posts on the iPhone support forums about this problem, but no mention of an official fix from Apple. One user claims that it could be related to Google Maps, which I use quite often. I'm going to watch this "other memory" usage to try to figure out which apps are increasing it.
I haven't seen a clear explanation of what is going on inside the iPhone that makes hacking applications very different from hacking the carrier, and from my experience building myPhone I can hopefully clarify this issue.
There are two CPUs inside just about all phones of any type. One is part of a GSM modem module that interfaces to the phone network, the other is the application CPU that runs the user interface (e.g. OSX on the iPhone). For the Gumstix Goliath a Siemens MC75 module interfaces to the SIM card, and the GSM antenna directly, and has USB, serial and audio interfaces. The way all these GSM modems are managed is by sending it command sequences that would be familiar to any old-timers, its a derivative of the Hayes AT command set. Inside the iPhone is a similar module. The GSM modem has a low speed low power CPU with its own flash memory. The service providers don't like having people mess with this firmware, because it controls the wireless signaling, power levels and cellular communication protocol directly, and they have to be certified for connection to the GSM network. For our homebrew phones we buy Telit or Siemens modules off the shelf and don't mess with their internals, and they just work. Thats why we can build usable homebrew phones.
Since this firmware interfaces directly with the SIM card, it can contain code that locks the phone to a particular service provider. The iPhone's GSM modem firmware is locked to AT&T. In order to modify this firmware the unlock applications such as iPhoneSIMFree had to take advantage of a debug feature or security flaw to load modified firmware into the GSM modem. Since the GSM modem is a standard component, hackers were probably able to find out its supplier and part number and get hold of a lot of documentation on how it works.
With the 1.1.1 release it appears that Apple closed off some security holes that were being used to hack into the iPhone, but also digitally signed the code inside the phone. This means that any attempt to modify OSX or the GSM firmware breaks the signature, and since the CPUs have hardware support for digital signatures, its extremely difficult to get around this form of protection. Its now several weeks since 1.1.1 was released on the iPhone and iPod Touch and it hasn't been broken into yet. The hackers have to find and exploit a completely new security flaw to get into the application CPU first, then find a way into the GSM modem. It is then easy for Apple to figure out how they are getting in, and fix it in their next release. With Apple releasing monthly updates, there isn't a viable business proposition based on hacking iPhones.
I still think Apple has a lot of unreleased iPhone software in the pipeline and will open up to developers when it has pushed out a few more updates. I think the ideal audience for an iPhone developer launch is MacWorld San Francisco in January. The highest density of startups and developers is in the Bay Area, and its the anniversary of the iPhone launch. Thats as near as I'm getting to a prediction....
In the meantime, the developers who want something to do could join the OpenMoko project, help build applications for it and maybe even port the Linux based OpenMoko to the iPhone hardware. Thats what I will be working on when I get myPhone up and running....
Wednesday, September 05, 2007
Tuesday, July 17, 2007
GSM/GPRS/EDGE, GPS, analog audio in/out, LCD, touchscreen controller, USB host port, 3D-accelerometer, NiMH battery charger (batteries not included)
|60-pin Hirose connector to verdex motherboard only|
|105.5mm x 67.3mm|
I've placed my order (I already have a Gumstix Verdex CPU board), and I'm still working on myPhone case design for this board and its 4.3" touchscreen LCD. Unlike the iPhone, the hardware and software is completely open, you can do anything you like with it, and thats also the problem, the software is extremely basic at the moment. However, its about the journey, not the destination, I'm going to have a lot of fun as we build this device.
The biggest suprise is just how good video looks. Everyone I show it to goes "wow". This is something you have to see in person. The video quality for "video iPod" mode and for "WiFi YouTube" mode is basically perfect. When running on EDGE, YouTube downloads a lower quality version of the video that has visible artifacts, and looks more like typical video on a PC quality (e.g. similar to my Slingbox). In any case this is a seriously good video player, and I happily watched several complete TV shows on it.
The built-in speaker is also surprisingly good, considering the limitations of a mobile device. I left the iPhone lying on the table and used it as background music.
The user interface is easy, no-one has any problem using it immediately, and typing on the keyboard is much easier and more accurate than I expected.
My main disappointment is with Email. It works, but I've got used to the Gmail client application on the Treo and Blackberry. The generic pop client just doesn't work well enough since it doesn't understand archiving and threading, and the safari browser version is inconvenient and slow when using EDGE. I really hope that the Gmail application is in the works for a future update, it seems likely, given the relationship between Google and Apple, and the Google Maps application that is already included.
For use at work, it can't yet replace my Blackberry. The iPhone VPN doesn't work with the VPN where I work. The LEAP security mode we use for wireless is also unsupported, although it works fine on my MacOS laptop. Using a simpler guest wireless setup I did see over 3Mbits/s in a speed test which is very impressive for a mobile device.
I can use Safari to get at Outlook Web Access, and http redirect based secure access to internal web pages worked fine. I have had no crashes or problems with Safari, trying it out on several sites.
For detailed insight into the iPhone, go read RoughlyDrafted, its by far the most thoughtful and interesting blog on the subject.
Monday, July 16, 2007
Go to http://rome.craigslist.org - there are a handful of entries, but the most important thing is that the site is still in English. Check out http://paris.craigslist.org - same thing.
Now try http://roma.kijiji.it and http://paris.kijiji.fr - notice the difference? Its localized and there are a decent number of items listed. Now see how many times you can figure out http://[city].kijiji.[country] (remember to use the local name) and see just how many cities and languages there are. Remember to try http://shanghai.kijiji.cn too... There are also a few countries where eBay bought a local brand - http://www.gumtree.com for the UK, and http://www.loquo.com for Spain.
So its clear that eBay has spent the last few years building up a global localized classified advertising business that has wide coverage and a lot of listings. They have had English-speaking Canada for a while (they also have French Canadian), so adding the USA must have been a small amount of extra work and cost, and there was a well crafted and debugged product ready to roll.
It may take a while to catch on in the USA, but its not likely to go away. Its free for the users, and can generate revenue by using services like banner ads and Google Adsense to monetize the page views. Kijiji is already doing quite nicely in my opinion....
[Disclosure: I used to work for eBay, I know some of the people who created Kijiji, but I have had nothing to do with Kijiji since it launched in 2005, and this is all public information]
Saturday, June 30, 2007
What about the open phones? Well, OpenMoko is running late, but is just about to ship developer phones for real, however the version that will have WiFi is their second revision that is months away, and their software is still being developed. However Gumstix are also very close to getting their new "Goliath" phone kit out, and that is a much more flexible design, where I can add WiFi etc. without replacing anything else, and I can wrap my own case design around the 4.3" LCD. The software is rudimentary, but we intend to port the OpenMoko packages to the Gumstix based "myPhone", and tinker with it to do whatever we want. In the meantime, I will soon have a phone that "just works" better than anything else....
Monday, June 25, 2007
There is a global commercial WiFi service that has good coverage in places like Hotels and Airports called Boingo. They aggregate services that are not branded under their own name, but if you have a Boingo account you can use the service. Last month they announced a deal to include FON in their network, more than doubling the number of access points to a few hundred thousand. This month they announced an annual flat rate plan for $39 per month. Very cool! Global WiFi access at a low flat rate.
I'm posting a series of updates at Millicomputing.com that describe a detailed design for an Enterprise server Millicluster. Its a radical improvement in terms of performance per watt and performance per dollar for some kinds of applications.
Back in January, I wrote a summary of thoughts on the iPhone from the perspective of typical product development practices. That post got a lot of traffic, it seemed to be very popular in Germany, where there is a great tradition of quality product design (I can't think of a German designed product that isn't well engineered).
Overall, with the additional advance information that has been released so far I don't think I was too far off the mark. I did expect that support for Adobe Flash and Java would be included, but they both appear to be missing. This is going to be the biggest pain, as there are a lot of web sites that use Flash or Java applets. Given time I expect these issues will be addressed.
I talked about Minimum Marketable Features (MMF) which are described in a book called Software by Numbers, written by my good friend Mark Denne. Its a philosophy of development that optimizes product features by releases over time. Many of the iPhone's shortcomings are explained by an initial focus on the needs of people who have never used a "smartphone" before, (rather than a focus on the business market) and a close integration with other Apple products that support an "insanely great" combination of product features.
For integration with non-Apple products, there is a two pronged strategy. One is to provide basic file viewers for Microsoft Word and Excel documents that are common email attachments. The other approach is to leverage Google's web based application suite within the Safari browser, which now includes Word, Excel and Powerpoint, Gmail etc, and also integrate with the Google maps application on the iPhone itself.
For the question of whether the iPhone can be used in corporate environments, there are two issues - virtual private network (VPN) access to get at internal web sites, and email integration with Microsoft Exchange. There have been some hints that the standard MacOSX VPN functionality is included, and Apple states that syncing is supported with Outlook/Entourage contacts and calendars. For actual Exchange email access, the Exchange server would have to support IMAP or POP (most aren't setup this way) or the Outlook Web Access client could be used in the Safari browser on the iPhone.
How many people have both iPods and cellphones in their pockets? This fall, how many students will turn up at college with an iPhone in their pocket?
My work machines are a MacBook Pro and a Blackberry. I'm building the homebrew myPhone design (more on that later), but I think I'm getting tempted by the user interface, I want an iPhone....
Saturday, June 23, 2007
The incentive in the short term is that you can get a high quality router at a very low price. FON has been giving routers away to seed markets and to surround Starbucks stores. The incentive in the longer term is that if you have a FON router at home, you can use any FON router you find on your travels for free.
I've had a FON router for some time. I met CEO Martin Varsavsky when he gave a talk at eBay last year, and got one of the initial Linksys units with custom firmware. When their own hardware solution came out it had a much nicer feature set, so I upgraded to La Fonera as they call it. I've been using it as my primary home router for about six months now, and its an excellent design. Its been much more reliable than the other low cost routers I've owned (Netgear and Linksys) and works as well as the much more expensive Apple Airport Express that I've had for a long time.
FON just released a booster antenna "La Fontenna", it is designed to give 6.5dB of extra signal strength and project further out than the standard antenna. The idea is that you mount it on the window facing the street, and it covers a larger area. I just got one, and while it seems to work fine, it doesn't appear to be giving me as much signal strength or range as I expected. I wandered around outside with my MacBook using iStumbler's WiFi monitoring widget to track signal strength, and it showed a bit more gain than my Airport. I think I need to experiment a bit more with positioning La Fontenna.
So next time you need a router, don't just buy a normal one, join Fon and support world domination!
Friday, June 22, 2007
I just hired an engineer for my own group, but we have a lot more openings:
We are looking for a performance and scalability oriented Java/Oracle developer
Senior Performance and Scalability Engineer
My good friend Ed Wustenhof just joined Netflix to manage the System Administrator group, he's got two open positions, a storage oriented one:
Sr Unix Administrator - Storage Focus
and a Capacity Planning and Performance Tuning one that I've copied the full info for below.
Senior Unix Administrator - Capacity Planning & Performance Tuning
Design, Plan, Deploy and Support all aspects of Netflix's server and storage environment with a focus on Performance Tuning & Capacity Planning.
The SA team at Netflix is a crucial component to the success of the day to day operations of the company. We are a highly motivated and capable team that is seeking a new member that can take a key role in the provisioning and support of our unix/linux servers while ensuring performance stays within thresholds.
Being the second line of support and subject matter expert, you will take individual responsibility for resolving system, storage and application issues reported via the Netflix Operations Center. You will be expected to resolve tactical issues while providing structural solutions. Your area of expertise is performance tuning and capacity planning.
Part of the position will require travel to and from the data center (15-20min or ~10 miles), sometimes multiple times/week. On-call responsibilities, night and weekend projects are also to be expected.
The candidate will often be the technical lead on short term projects. They will be the Subject Matter Expert with its associated responsibility. Some level of project planning and coordination of resources is to be expected. This includes vendor and contractor coordination.
We are looking for someone with the following technical experience:
•Deep experience of Unix with at least one of the following operating systems: AIX, Linux or Solaris.
•Expert in capacity planning and performance tuning.
•Strong experience in web based environments and supporting such architectures such as Java, Tomcat, Apache and other web applications.
•Knowledge of IBM enterprise/midrange systems and storage virtualization is a plus.
•Strong coordination and analytic skills.
•Project planning experience and the ability to manage multiple projects simultaneously.
•10+ years IT experience with at least 8 years as System Administrator.
•BS or MS degree in Computer science or equivalent experience.
Exposure to the following highly desirable:
•Experience with Symantec/Veritas product like VxVM and Netbackup
•Strong expertise with SAN-based storage/application required.
•Large Scale (5Tbyte +) Storage solutions deployment experience.
•Unix shell and Perl programming.
Tuesday, June 12, 2007
About a year ago I gave a paper at an IEEE E-Commerce conference that described how I used the latest pi-calculus enhanced version of the old Inmos Transputer language Occam to build a very efficient massively threaded simulator for large scale p2p web services that ran on my laptop.
I blogged this at the time, but The paper itself is subject to the rules IEEE sets out for peer-reviewed conference papers and you need to be a member or pay a fee to get it from http://doi.ieeecomputersociety.org/10.1109/CEC-EEE.2006.81
However, I wanted to share this more widely and have just posted a pdf of the presentation slides for my IEEE paper. Here are the first few slides, and a description of the Tools section:
Simulation of Skype Peer-to-peer Web Services Choreography Using Occam-Pi
Complex web services are very difficult to test and verify before deployment on a large scale.
A semantically equivalent in-memory simulation can be built using Occam-pi that runs up to 100,000 nodes about a million times faster than real life.
Rapid protocol development and detailed behavioral analysis using simulation supports the development of robust services that can scale.
The implementation of a simulator that models centralized web services and application to application messaging over the Skype Peer to Peer network is described.
- The Landscape
- The Problem
- The Tools
- The Solution
- Running the Simulation
The Tools – Pi Calculus
- WS-BPEL, SSDL and WS-CDL are based on Pi-Calculus
- Pi-Calculus is based on Communicating Sequential Processes (CSP)
- Pi-Calculus provides a formal model of parallel message based computing
- The Occam language is based on CSP, and has been extended to add the Pi-Calculus extensions to form the Occam-Pi language.
- The primary implementation of this language is known as KROC, the Kent Re-targetable Occam Compiler
- KROC is freely available from the University of Kent at Canterbury, UK
- Runs on Intel architecture Linux, MacOS X, and Microsoft Windows/Cygwin platforms. Older versions exist for SPARC, PPC etc.
The Tools – Occam Language Constructs
- The constructs that are used in Web Services choreography map directly to Occam language constructs
- The Occam language has direct support for sequential, parallel and alternate processing blocks, complex protocols, and channel based communications
- Occam-Pi adds more dynamic constructs to the language
- Mobile channels - pass a channel end over another channel
- Mobile processes - suspend a process, pass it over a channel and resume it in a new context
- Dynamic process forking with barrier synchronization
- Rigorous Occam-Pi Compile-time Checks
- Processes or expressions are not allowed to have any “side effects”
- Syntax and usage of all protocols, data and constructs is checked
- Occam is designed to allow very comprehensive static analysis
The Tools – Occam Runtime Characteristics
- There is no need to use XML message encoding or namespaces
- since the compiler can check that a protocol is being communicated correctly
- At runtime, the Occam-Pi language is fast small compiled code
- with its own threading model, in a single process
- The Occam-Pi runtime detects and reports deadlock
- including the line number in the code at which each process was stalled
- Occam-Pi is very efficient
- All communication takes place in a single address space at memory bus speeds
- Basic process creation takes 20 nanoseconds on an 800MHz PC
- Basic channel communications takes 70 nanoseconds
- Compared to typical web services transactions over the internet these transactions are about a million times faster
- The language is also very compact, and one hundred thousand to one million threads can be created within a 2 GByte address space.
- [Cygwin] “GNU + Cygnus + Windows”, http://www.cygwin.com/
- [GraphML] “Graph Modeling Language “, http://graphml.graphdrawing.org/
- [Hoare78] C. A. R. Hoare. “Communicating Sequential Processes.” Communications of the ACM 1978.
- [Hoare85] C. A. R. Hoare. Communicating Sequential Processes. Prentice Hall, 1985.
- [KROC]. “Kent Retargetable Occam Compiler” http://www.cs.kent.ac.uk/projects/ofa/kroc/
- [Occam2] Inmos Ltd, Occam 2 Reference Manual. Prentice Hall, 1988.
- [Milner99] Robin Milner, “Communicating and Mobile Systems: the Pi-Calculus”, Cambridge University Press, 1999.
- [Skype]. “Skype Developers API” https://developer.skype.com
- [SSDL] Webber et al. “Asynchronous Messaging between Web Services using SSDL.” IEEE Internet Computing Vol10, No1 2006.
- [WS-BPEL]. “OASIS Web Services Business Process Execution Language” http://www.oasis-open.org
- [WS-CDL] “Web Services Choreography Description Language”, http://www.w3.org
- [yEd] “Graph visualization and editing tool”, http://www.yworks.com/
Friday, June 08, 2007
Its available on Fractal's site at fractal.bebo.com and via the iJigg widget below.
Saturday, May 19, 2007
Friday, May 18, 2007
Its not easy to follow the advice, but it does give a clear vision of what to aim for, and how to know when you get there...
Thursday, May 17, 2007
I'm going to be at Maker Faire in San Mateo this weekend, helping explain the 3D printer to attendees at the Techshop pavillion, and hanging out with Matt on the SVHMPC booth. I'm on duty from 2pm onwards both Saturday and Sunday for Techshop so I should be easy to find and I'd be happy to chat about how to make mobile phone cases or millicomputing or whatever.
Wednesday, May 16, 2007
The full job specification is at http://jobs.netflix.com/DetailFlix.asp?flix1642
Here's what we are looking for:
- Designing and developing the Netflix web application and middle-tier architecture focused on personalization software.
- Full software life-cycle ownership of key strategic business logic from inception through production.
- Conceiving, articulating and implementing projects in coordination with technical and non-technical coworkers for product enhancement and improvement.
- Experience with high-traffic, highly scaling, multi-threaded software design.
- 7+ years of web application development experience. (Java a plus)
- Experience with Oracle or equivalent database system.
- Flexibility to shift priorities based on changing needs of the business.
- A Bachelors or Masters in Computer Science or equivalent engineering.
- Strong customer orientation, both internally and externally.
- Strategic thinking both technically and business-wise.
- Ability to execute successfully on business strategy.
- Self motivated / self starter.
- Personal drive to achieve world class software development.
- Desire to work in a fast-paced, evolving, growing, dynamic environment.
- Love of technical challenges and the sense of pride it brings to solve them.
- Ability to work well in a cross functional environment.
Friday, May 04, 2007
This seems like a nice, high performance multiplatform way to build web based user interfaces. It should give Flash some competition...
I'm handing back eBay's Dell D600 with Windows XP and getting a MacBook when I start at Netflix, it will be nice to have XP running only when I need it, and when the Silverlight version of Netflix web video ships, it will support Mac.
Joost has also released a new beta build, however I don't have a machine I can run it on during this transition. They don't support my old PowerBook G4 yet....
Monday, April 30, 2007
Here are some job descriptions, let me know if you apply to them...
Friday, April 27, 2007
Tuesday, April 24, 2007
I've also been continuing to develop the general purpose millicomputing concepts and have been documenting them on my companion blog. I have bought a Gumstix module for use in a phone, and I'm working on benchmarking it at the moment.
Saturday, April 21, 2007
For a detailed explanation of the algorithm behind the metric, Neil Gunther has posted a series of articles that show how Load Average is a time-decayed metric that reports the number of active processes on the system with a one, five and fifteen minute decay period.
The source of the number of active processes can be seen in vmstat as the first few columns, and this is where Solaris and Linux differ. For example, some Linux vmstat from a busy file server is shown below.
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu---- r b swpd free buff cache si so bi bo in cs us sy id wa 4 43 0 32384 2993312 3157696 0 0 6662 3579 11345 7445 7 65 0 27
The first two columns show the number of processes that are in the run queue waiting for CPU time and in the blocked queue waiting for disk I/O to complete. These metrics are calculated in a similar manner in both Linux and Solaris, but the difference is that the load average calculation is fed by just the "r" column for Solaris, and by the "r" plus the "b" column for Linux. This means that a Linux based file server that has many disks could be running quite happily from a CPU perspective but show a large load average.
The logic behind the load average metric is that it should be a kind of proxy for responsiveness on a system. To get a more scalable measure of responsiveness, it is common to divide the load average by the number of CPUs in the system, since more CPUs will take jobs off the run queue faster. For disk intensive workloads on Linux, it may also make sense to divide the load average by the number of active disks, but this is an awkward calculation to make.
It would be best to take r/CPU count and b/active disk count then average this combination with a time decay and give it a new name, maybe the "slowed average" would be good?
Monday, April 16, 2007
Meanwhile, Rich has been trying to make a multi-platform (Solaris, Windows, Linux, FreeBSD, OSX, HP-UX, AIX) version of SE for a long time, and finally gave up trying to implement his own language, and based his latest development, the XEtoolkit, on Java 5. The first full release XEtoolkit 1.0 came out on April 15th, 2007. The code is released and supported under both open source and commercial licenses, by Rich's new company - Captive Metrics. The GPL license allows full free use of the provided tools, and development of new and derived tools that are also contributed to the community. The commercial license allows custom XEtoolkit development for proprietary tools, with a higher level of support.
The XEtoolkit 1.0 release doesn't support HP-UX or AIX, but AIX support is coming soon. I encourage you to try it out, give Rich some feedback and make it worth his while to continue. He's one of the very best programmers and performance tool architects I've ever met....
Thursday, April 12, 2007
More pictures of the latest myPhone 2.0 case design.
This version has a deeper case, and was printed at a simple angle, the previous attempt ended up warping, so I hope these changes will keep it flat. It also has a slot in the bottom end to take an iPod connector, which carries power, USB, stereo line level output etc. an antenna mounting hole at the top, and a retaining clip design to hold the LCD in place.
This contains 2.6 cubic inches of ABS plastic, and used 1.4 cubic inches of support material, which costs about $40 in materials and took 8 hours to print at Techshop.
Wednesday, March 28, 2007
While the current mainstream CPUs were driven by the development of the home PC market, this generation is driven by the development of the mobile, battery powered device market, which is a very large. For example the worldwide cellphone market is something like a billion devices a year.
I think that there could be some interesting general purpose computer systems built from low power devices (CPUs that use less than one watt). I looked around but wasn't sure what to search for... I do know about the systems that are sold for embedded use, but they are typically configured using lower speed and lower memory options.
Does anyone know of vendors selling general purpose systems, or a category name for this space?
[Update: I asked around, and thought a while, and decided that this is interesting enough to have its own name "millicomputer" and its own blog "millicomputing"].