Pure mudslinging about conflicts of interest based on things that don’t seem to be true. I won’t mention the mudslinger, but I can tell you that if I bothered to read that other blog — which I don’t — I would unsubscribe.

Build your brand on merit like InsideHPC did, not on publicity stunts.

PDC’s are quite valuable to attend (access to Microsoft product teams, exposure to their roadmap, opportunity for light-bulb” moments, etc).  That said, I may decide not to go after the session list is released – a simple balancing of priorities.

Anyway, I’ll likely keep my registration – I would actually love it if Microsoft could change my plans about Azure this October.

Are you going? Or not? If so, please share your reasons.

• The intrepid John Powers dives into it more in his post, Digipede Network 2.4 — Beyond the Press Release.
• Also, Michael Feldman of HPCwire wrote about the release here Digipede Sticks to Its Grid Computing Roots. 
• The press release (this is a link to the PDF).

Those paying close attention might ask "what happened to 2.3?"  The answer is Digipede trivia.

• Part of a failed experimental branch? No. 
• Is 2.4 actually numbered 2.3.1 under the covers? No. (A minor dig at Windows 6 R2).

The actual reason dates back to the days when .NET 2 was released.  Back then, we were ready to release Digipede Network 1.1 with .NET 2 support.  To avoid naming confusion with .NET 1.1, we decided to skip the “.1” and went straight to “.2”.  Was it in fact less confusing?  Probably not materialy.

So, why no 2.3?  It is an ever so slight (and obscure) homage to those early days: for .NET 4 we decided to release something that ends in “.4”. 

Like I said: trivia.

• Model-first development.
• Lazy loading through relationships (i.e., no longer have to call Load)
• POCO (i.e., define your own data classes against a model).
• POCO only (i.e., define the model fully in code).
• Code Generation options using the new T4 facility of VS 2010.
• Testability improvements through IObjectSet
• Can override SaveChanges
• Better disconnected workflow (both by writing a little code and a no-code option that uses a different code generator).
• Much better SQL (more compact, more efficient)
• Execute arbitrary SQL
• Easier Stored Procedures
• Functions (a little strange how this was implemented, but now they are available).
• Foreign Keys in the entities (no more manual interpretation of the Reference!)
• Better Binding for forms apps and WPF

Pandelis, what do you think?

It is the openness of this platform, the ability of developers to mix and match the different components, and to do it between the cloud and in-premises solutions that makes this such a winner. 

This last point is an important one.  Microsoft is in a unique position to help enterprise IT bridge to the cloud.  While I don’t think Amazon and Google will cede that market to Microsoft, their current offerings aren’t a natural fit. 

The offering was rich then, but since then Microsoft has continued to push these offerings forward dramatically. 

At the time, my biggest concerns were the one-size-fits-all approach to their provisioning model and their lack of full trust (two things that could make it harder to deploy the Digipede Network onto Azure).  Today those issues have been taken off the table and help support many more use cases, opening up Azure even more to non-Microsoft technologies and fortifying the extremely important IT bridge.

So what are the improvements in openness?

Allowing full trust opens up the door to, well anything.  Unmanaged code,  PHP, MySQL, Java, TomCat, etc. can all run on Azure.  Matt Mullenweg of Automattic demonstrated a WordPress instance running that way.  Kind of anti-climactic, because it would have been a big deal if wordpress.com was moving to Azure.  Simply running a WordPress instance isn’t really that interesting.

Custom VM images are also coming to Azure which will make it much easier to put whatever you want on a VM and deploy it efficiently.

For IT?

Too many items here to enumerate.  SQL Azure integrating into SSMS; Azure integrating into MOM; SQL synchronizing with cloud instances; (this list really does go on and on . . .).

Another important part of this IT bridge?  Not Microsoft’s new App Server, AppFabric.  Though I am excited about this – it is something that has been missing from the Microsoft stack – the key point here is that it runs on premises and in Azure.

Conclusion?

These new features in Azure push Microsoft out even further than the other cloud vendors.  No one else has the depth and breadth in tool support and service offerings.  No one else is innovating so quickly on so many parallel fronts. 

Will Amazon and Google cede the space?  Of course not, but I think they’ll need  to reposition their cloud brands.

I like those guys.  They are smart and do excellent work.

Anyway, they released the sample earlier today.  From their site:

As of version 1.5 (which is currently in beta), the world’s coolest spreadsheet can use Digipede Network grid computing to distribute and execute workbooks in parallel. The example on the Exchange is based on the excellent IronPython sample created by Robert W. Anderson of Digipede. The Digipede Network is a brilliant way to get distributed, parallel computation on Windows. It only took a few minor changes to convert Resolver One to run on the Digipede Network and to get the IronPython sample to execute Resolver One workbooks.

Giles gives some more background to the path that got us here on his recent post, Resolver One and Digipede.

The combination of our two products offers a pretty elegant solution.  Like I said before,

Try doing that with a spreadsheet or grid that isn’t based on .NET . . .

. . . like Excel and Windows HPC Server.  No, don’t. Trust me.  It is really hard, complex, and brittle.

They used my IronPython Worker sample and customized the front-end Python code, leaving the C# adapter as-is.  With very little coding they had an elegant grid-enabled spreadsheet.  Try doing that with a spreadsheet or grid that isn’t based on .NET . . .

Giles said they will have support for this in Resolver One 1.5, coming out in the next couple of weeks.

Very cool. 

I’ve just installed Resolver One to take a closer look.  Already I’m impressed, but I’ll leave that for a future post.

I have been playing with IronPython a little.  With the release of Digipede Network 2.2, I am now able to post the sample I wrote.  It shows how to distribute IronPython objects on the Digipede Network.  You can find it on the Digipede community site.  See the posting there for details and download instructions.

The sample uses IronPython 2.0.1 and the included version of the Microsoft Dynamic Language Runtime (DLR).  While I focused on IronPython in this sample, it would be pretty easy to expand it to support other DLR-based languages.

Comments welcome.  I am specifically interested in feedback on DLR integration and initializing ScriptScope objects for each worker thread.  It seems that I should be able to do some of this only once at global scope.

By the way, one thing I like about this sample is that it shows how to keep user code completely de-coupled from the Digipede Network while still taking advantage of our deployment and payload distribution model.  This has always been supported by the Digipede Network, and this makes a good example.

Looks like it will be an interesting program.

If you are going and want to meet up, email me at robert at digipede dot net.

The first thing I did get our old Python sample running in IronPython.  This was the first user-contributed sample (thanks to Sean True).  That sample (see it here) uses Python COM libraries to invoke a job with the Digipede Network COM APIs.  This didn’t submit objects, just executed a command-line application.

I’m happy to say that the code required very little modification to run under IronPython.  The only difference is in the syntax of the “import” commands.  Kudos to the IronPython team.

I’ll post the working code once I get a little farther.

Next step: distribute IronPython objects.  Fairly straightforward, but I’ll write a reusable C# Executive to load the IronPython class definition.

Of course, now begins the discussion of how all these pieces fit together.  

This is not a simple approach like Amazon’s EC2 or Google App Engine.  Not to trivialize either, but they are certainly easier to understand.  Try explaining them to the proverbial grandmother — no problem, especially if you leave out virtualization and pythons   (preemptive comment: I know AWS is much more than EC2 and that bigger and better things are coming from Google).

Regardless, the Microsoft Azure is multi-faceted.  In typical Microsoft fashion, there is a lot for a developer to choose from:

• Azure Storage, Management, and Compute.  Run WCF/ASP.NET based services, with work queues and data storage.
• Microsoft .NET Services, nee biztalk.net (wrote about here).  This gives you an Internet Service Bus, Access Control, and Workflow Services.  Messages and workflow in the cloud connecting other cloud and enterprise offerings.  Very big deal.
• Microsoft SQL Services, nee SQL Server Data Services or SSDS.  Eventually a relational model in the sky, currently not too different from Azure Storage.
• Live Services: Not too much detail on this today, but this is clearly what was “Live Mesh”: a rich synchronization framework, “live operating environment” for writing applications to across the Web and on user’s devices. 
• Windows Live (Live Office, Live Sharepoint, Live Dynamics CRM, etc). In-cloud applications extensible by partners and users with in-cloud and in-premises solutions.

It all does fit together, and will be of immediate value to developers.  As Marc Jacobs of Lab49 said to me afterward,

We could make use of all of these services today.

Damned straight.  It is the openness of this platform, the ability of developers to mix and match the different components, and to do it between the cloud and in-premises solutions that makes this such a winner. 

This last point is an important one.  Microsoft is in a unique position to help enterprise IT bridge to the cloud.  While I don’t think Amazon and Google will cede that market to Microsoft, their current offerings aren’t a natural fit. 

Taking this all together — not forgetting Microsoft’s leading developer productivity story — it looks like a home run to me.

Google releases a new browser.  The world declares “browser war” with some apprehension and  relish.  Web developers are cringing because browser compatibility is a major source of effort, cost, and frustration for software developers.

Q. Why would Google do this to us?  Just to take away Microsoft browser share? 

A. No.

Q. Are they doing this to extend the “Google OS” to the desktop in a way they control?

A. Probably, but that isn’t even their first concern.

Q. So, what is going on?

A. Well, I’m glad you asked.

Google is working to make their JavaScript-view of the Web as powerful as possible.  This makes sense given their enormous investments in JavaScript and in their own application suite.

Contrary to the approaches of Microsoft and Adobe with their Rich Internet Applications (RIA) frameworks, Google has focused on JavaScript. Where Microsoft and Adobe are building a better user experience inside of a container, Google is creating a better user experience through dynamic HTML and AJAX techniques.

Their developer model includes building out tooling to make it easier to author AJAX applications.  This includes the efforts made in the Google Web Toolkit (GWT) to enable modern IDE tooling for AJAX development. This allows developers to build maintainable object-oriented applications (in Java) that get converted and optimized to JavaScript.  Plus it promises cross-browser compatibility.

On the client side, they have Google Gears to enable local storage, improved caching support, and offline mode.

Q. So what have they been missing?  A browser? 

A. Not exactly.  They’ve been missing a JavaScript client runtime engine.

Google has made great advances in AJAX application development and tooling, but they have had to rely on others to provide reliability, responsiveness, performance, etc.

And that is what Chrome is about: taking control of the runtime engine for Google applications.  This makes the Google applications way more compelling.  More specifically, Chrome is about delivering that engine.  As Google says, they would love it if other browsers adopt the engine too.  I buy that.

Of course, by that time Chrome will be differentiated from its JavaScript engine.  By then Chrome will be about the Google OS.

Version 2.6 “Tyner,” named for jazz pianist McCoy Tyner, contains a number of new features that make WordPress a more powerful CMS: you can now track changes to every post and page and easily post from wherever you are on the web, plus there are dozens of incremental improvements to the features introduced in version 2.5.

These feature changes are actually pretty big.  Revision tracking?  Support for Google Gears?  Full support of SSL (finally)?  Theme previews?  Really cool “Press This” button?  Big.

This feels to me like a major release.  Probably not as major as the 2.5 release, but still pretty major.

In my book, 2.5 should have been version 3.0 and this one should have been 3.5.

Does the version number matter?  Yeah, it does.  It isn’t just about marketing.  It signals something about the maturity of the product.

Disclaimer: I am not immune to such version number mistakes.  After all, the Digipede Network 2.1 should have been version 2.5.

Note that I refer to cloud services, not to the cloud.  I am not interested in defining cloud as a term, because I don’t think it very useful.  For those of us in the distributed computing space, cloud is the latest buzzword to compete with the word grid in terms of utter ambiguity.  I think the ship has already sailed on this one and I’m not going to try to call it back.

So, everyone is talking about cloud services and much of the conversation centers on understanding them and how they are changing the landscape.  Of course, cloud services are not one thing.  I find it helpful to think about them as parts of a continuum.  This seems useful regardless of the technical level of the people with whom I’m speaking.

The diagram to the right shows this continuum from infrastructure to platform to software.   Brief definitions of these parts are:

• Infrastructure includes provisioning of hardware or virtual computers on which one generally has control over the OS; therefore allowing the execution of arbitrary software.
• Platform indicates a higher-level environment for which developers write custom applications.  Generally the developer is accepting some restrictions on the type of software they can write in exchange for built-in application scalability. 
• Software (as a Service) indicates special-purpose software made available through the Internet.

I have indicated several companies that play at different parts of this stack.  This list is not comprehensive nor does it attempt to represent motion across the stack.

One scenario in which I find myself talking about the continuum is when people equate Amazon EC2 with Google App Engine.  EC2 is a flexible / scalable virtual hosting platform with provisioning APIs.  It allows you to dynamically scale the number of instances of your OS (i.e., Linux).  What you do with those instances is up to you.  Google App Engine operates at a much higher level in the stack.  It is a new software platform with specific APIs.  It requires developers to build for this specific platform.  yes, they are both in the cloud, but they are very different services. 

Another scenario in which the continuum is useful is in thinking about what vendors and new entrants might be up to.  The continuum makes one thing even more clear: many vendors that operate higher in the stack are relying on their own internal lower-level infrastructure or platform.  This begs some questions: which vendors will expose lower-level interfaces?  And of course, which vendors will move up the stack? 

• SalesForce is already moving down with their PaaS offering. 
• Any chance Google will expose its infrastructure stack?  I doubt it, but I do expect them to move down a little. 
• Some of the readers of this blog probably know better than I where Amazon and Microsoft are planning to go.

Yet another way it is useful is in comparing vendors inside of a particular category.  Maybe I’ll write more on that later.

Is the continuum obvious?  Using the definition of obvious from patent law, yes, but I think it a useful paradigm.

I am definitely excited about this release.  While Microsoft is not breaking new ground here, the addition of a distributed cache to .NET is a great addition to the platform.  Certainly there are competing technologies, but Velocity will be a very simple choice for developers and ISVs because we’ll be able to count on its availability. 

This ISV is interested, so we tried it out.

We have many customers who use our Executive pattern to load and cache job-specific data for compute-intensive jobs on the Digipede Network.  These data are often fetched through WS calls or directly from SQL databases.  Often this is performed in the Executive.Start method.  Before Velocity, the code might look like this:

protected override void Start() {
    // read the CBOData object from the database.
    _cboData = ReadCboData(cache.Get(JobTemplate.Parameters["CBODataStore"].Value));
}

Including Velocity in this example is really easy.  The following snippet adds use of the Velocity cache:

protected override void Start() {
    // get cache
    CacheFactory factory = new CacheFactory();
    Cache cache = factory.GetCache("CBOCache");
    // see if our CBOData object is already there
    string key = JobTemplate.Parameters["CBODataKey"].Value;
    _cboData = (CBOData)cache.Get(key);
    // if not, read it from the database.
    if (_cboData == null) {
        _cboData = ReadCboData(cache.Get(JobTemplate.Parameters["CBODataStore"].Value));
        // store it in the cache for later use
        cache.Put(key, _cboData);
    }
}

With a few lines of code, we reduce the load on the database server and network and spend more time computing.  (I’m making an assumption with this simple code that all Executives don’t start at once, an assumption made obsolete by seeding the cache from a master application).

Of course, this is a simple example, but there are many other use cases.  For example,:

• Digipede-enabled applications can share results; 
• master applications can load the cache with job-specific data; and,
• others where baking Velocity deeply into the Digipede Network start looking pretty interesting.

I have seen many posts on “must-haves” for a Velocity RTM.  I mostly agree with the lists I have seen.  I’ll have a list too mostly from the ISV perspective.

Cool stuff.

Dan has a write up of some of the enhancements added to this release here.  He said we should have probably called it 3.0, but it is really more of a 2.5.  We’ll be hosting webcasts soon going over the new features.  

Thanks to the team for all the hard work in getting this out the door.

Follow http://twitter.com/010111011010111 for Digipede announcements.

Come on by and see us.

BTW: I won’t be at this event, but I’ll be at the one in SF on March 13th.  No Digipede booth or give-away there.

. . . 2 worlds:

1. The world as we think about it.
2. The world as we can manipulate it.

The difference between these two is what is called the semantic gap. 

This is a great working definition. 

Jeff writes about this specifically regarding PowerShell and instrumentation providers and asks the question,

So why do instrumentation providers close or not close the semantic gap?

Yes, some do, and some don’t.  This isn’t just about hierarchy of needs, but also about prioritization.  How important to the provider is a narrow semantic gap for product X when used through interface Y? 

In the case of X := Digipede Network and Y:= PowerShell, we thought it pretty important.

But how do you decide if narrowing the gap is worth it?  Engineering costs aside, understanding what your interface could look like in PowerShell can help you decide.  Internally, we answered these questions:

1. What would a PowerShell script look like just using your .NET or COM APIs? 
2. What could it look like with Cmdlets? 
3. Would these Cmdlets support how we think about the Digipede Network (i.e., small gap?).

I already said the answer to #3 turned out to be yes and in a previous post, I gave an example of the gap in Why a SnapIn for the Command-Line?  This example highlights the gap for a common operation on the Digipede Network: get the description of a pool of resources.

If you are thinking about supporting PowerShell in your product, take a look at my post.

I hope this helps you decide.

Thanks to MVP Marco Shaw for the idea. 

Getting the logo was arduous.  This has less to do with the technical logo requirements and more to do with the complexities of the process itself.  Some of the complexity is inherent in such a process, but much was due to the program itself being a sort of “work in process”.  But hey, that’s why we early certifiers got the testing fees waived.  I think those who begin the process now will find the test requirements and tools are better written and more robust.

As I said, passing the technical requirements was not arduous for us (we were already very close), but passing the tests did require some minor improvements to the Digipede Network that were motivated by the test:

• Support for User Account Control (UAC).
• More useful logging on the Digipede Server and during installations.
• Improved user messaging and event logging during error conditions between server components and the database.
• Improvements to the Installation Guide including new sections on Custom Actions, installation artifacts, and more.

Some of these changes have already made their way into the shipping product, though others won’t be available until the Digipede Network 2.1 (which, while a minor upgrade, contains many features beyond the improvements mentioned above — I think the feature set will be announced soon).

So now we’re ready for the big launch of Server 2008, Visual Studio 2008, and SQL Server 2008 in Los Angeles on February 27th.  If you are going to be there, come see us at the Partner Pavilion.  I’m pushing for some kind of Digipede swag — but I’m not in marketing .

It is amazing to me the level of religious fervor that Microsoft still inspires. The bashers out there can be perfectly calm and reasonable about a wide range of topics – but say the word “Microsoft,” and they turn bright red and irrational. I have watched this phenomenon for years, and still find it inexplicable. Microsoft is a company. That company makes software. Some of their software is very, very good. Some of it is remarkably bad. I don’t understand why some people find it so hard to remain objective (or even civil) when discussing their products and market presence.

Statement of fact.  Nothing new exactly, but then he goes on how this relates to our company (emphasis mine) . . .

Many Microsoft bashers think that all of us at Digipede are mouthpieces for the Evil Empire, and that we are just pawns of the Microsoft machine. On the other hand, while we have plenty of fans within Microsoft, there are also some Microsoft employees who think we are difficult annoying troublemakers . . .

Pawns of Microsoft? Please. But maybe we are misunderstood. The other day I found myself convincing Steve Gillmor that I’m not a Microsoft fanboy (I don’t think I succeeded). But John says it well,

In fact, none of us at Digipede love or hate Microsoft – we work with Microsoft. We do so for real-world business reasons that help us change the world for the better while building a great company. We work with other companies too, but Microsoft occupies a special place in the technology landscape, and we work very, very hard to understand how to work with them to our mutual benefit. There are some great people there doing great things, and the bashers only hurt themselves by blinding themselves to these very real contributions.

Yeah, I don’t love or hate Microsoft; however, I do really like Microsoft .NET.  Does that make me a pawn of Microsoft?  .NET isn’t my religion.  I’m not a zealot about it.  It doesn’t mean that I think everyone should be using .NET / Windows nor does it require that I go around bashing Apple / Sun / IBM / Google / Linux / Java / PHP / Rails / whatever. 

I don’t think Microsoft bashing is a requirement for entry to the HPC and Apple fan clubs either, is it?

As long as I can remember, I have detested the religion of the OS (or programming language, or platform, etc.). Passion for technology is great — it’s a requirement for success in this field.  But I’m tired of people using their passion to bash, bash, bash. 

Maybe I just don’t get it, but if you’re a basher, please just move along.  And if you still think I’m a Microsoft pawn, well, I’m not going to convince you, now am I?

Note to Robert Scoble: you had the temerity to criticize Apple and the zealots came out in force — even called you a Microsoft shill. Welcome back to our club, Robert — though I’m not sure you ever really left.

1. Microsoft announced version 2 of Compute Cluster Server, renamed Microsoft Windows HPC Server 2008.  Ever since I saw a preview of this a couple months back, I have been excited about some of the enhancements they’ve made in terms of manageability and interactivity.  Mostly, I’m excited about new interop scenarios between the Digipede Network and HPC Server.  As some resources become available, I plan to do an internal proof of my idea — I hope I’ll have more on this soon.
2. Microsoft announced their Parallel Computing Initiative with the mission of enhancing developer productivity for multicore and distributed systems.  Most exciting (to me) is the announcement of their Parallel Extensions to the .NET Framework.  These extensions include mechanisms for expressing parallelism inside managed code.  One thing that will be built on top of these extensions is P-LINQ.  Expect previews of this technology will begin to come out over the next six months.  Why am I excited about this?  Because the more tools that are made available to .NET developers to express parallelism, the easier it will be for the Digipede Network to fit in and managing varying workloads across a large number of disparate computing resources.

Note to Wagg-Ed:  there is no such thing as Visual Studio 2007.

One of the demos includes using Excel as a front-end to a distributed code-behind calculation.  A common follow on question was asked (heavily paraphrased),

what about code trapped in Excel workbooks?

The answer is, yes, we can (and have several customers that do) distribute Excel workbooks for distributed execution.  Our software moves the workbooks, launches Excel through COM, executes calcs in the workbook, saves them, and brings back the workbook. 

The prospect says,

That is both commendable and horrifying at the same time.

Heh. 

As is most of the work we’ve done to provide stable, registration-free COM contract for the Digipede COM users.

One question I didn’t get into is, why write a SnapIn when PowerShell supports full scripting for .NET APIs?  The short answer is that we can provide a better command-line experience with PowerShell cmdlets than the “out of the box” PowerShell experience.  Windows PowerShell Architect, Jeffrey Snover, did a great post on this a few months back: Cmdlets vs. APIs. 

I thought I’d give an example of the difference as it relates to the new Digipede SnapIn. 

For example,  suppose you want to enumerate the pools on a Digipede Network.  Natively in PowerShell, you can do the following:

[System.Reflection.Assembly]::LoadFrom("C:\\Program Files\\Digipede" +
  "\\Framework SDK\\v2.0\\Net2.0\\Digipede.Framework.dll")
$DigipedeClient = New-Object -TypeName Digipede.Framework.Api.DigipedeClient
$DigipedeClient.SetUrlFromHost("MYSERVER")
$DigipedeClient.Management.GetPools() | ft

That is pretty verbose (note the first line is split in two to improve readability in my blog). This same operation becomes much simpler with the Digipede SnapIn:

Add-PSSnapIn DigipedeSnapIn
$DigipedeClient = New-DNClient -Network MYSERVER
Get-DNPool

And what about the output?  Without any extra information, PowerShell will format output on its own — a very cool feature.  The former example produces a default table output:

IsReadO  PoolId Name    Descrip Collect Schedul Potenti TotalCo JobTota TaskTot
    nly                 tion    ion     ingAlgo   alGhz mputeRe lComple alCompl
                                          rithm         sources     ted    eted
-------  ------ ----    ------- ------- ------- ------- ------- ------- -------
   True       1 Mast... Mast... {Mas... ...rity      66      19     665   57584
   True     601 Fifo            {Mas... ...tOut      18       3       0       0

PowerShell can’t determine which properties should be included, so it makes a guess.  The DigipedeSnapIn includes formatting information to give much more readable (and useful) default output:

PoolId Name                 Description                    Resources Potential
                                                                          GHzH
------ ----                 -----------                    --------- ---------
     1 Master Pool          Master Pool                           19        66
   601 Fifo                                                        3        18 

So, the SnapIn definitely provides a better command-line experience.  And the great thing?  These two worlds are not mutually exclusive.  For example, suppose you want to specify the default pool for a DigipedeClient object?  We haven’t exposed that as a property on New-DNClient nor provided a cmdlet for this purpose, but the API includes the PoolId property.  Just do this:

$DigipedeClient.PoolId = (Get-DNPool -Name "Fifo").PoolId

Now, the default pool is a pool named “FIFO”.

This brings me back to something I wrote in my previous post:  PowerShell is cool.