rwandering.net
The blogged wandering of Robert W. Anderson
April 6, 2009 at 6:31 am · Filed under .NET, Grid Computing
. . . or IronPython-ipede (Part II).
I have been playing with IronPython a little. With the release of Digipede Network 2.2, I am now able to post the sample I wrote. It shows how to distribute IronPython objects on the Digipede Network. You can find it on the Digipede community site. See the posting there for details and download instructions.
The sample uses IronPython 2.0.1 and the included version of the Microsoft Dynamic Language Runtime (DLR). While I focused on IronPython in this sample, it would be pretty easy to expand it to support other DLR-based languages.
Comments welcome. I am specifically interested in feedback on DLR integration and initializing ScriptScope objects for each worker thread. It seems that I should be able to do some of this only once at global scope.
By the way, one thing I like about this sample is that it shows how to keep user code completely de-coupled from the Digipede Network while still taking advantage of our deployment and payload distribution model. This has always been supported by the Digipede Network, and this makes a good example.
Tags: .NET, Digipede, DLR, IronPython, Python
March 3, 2009 at 6:46 am · Filed under .NET
Some time ago I took a quick look at Patrick Smacchia’s NDepend. While I was impressed with it, I never took the time to dive into it. Recently, I took another look to see if it could help inform some refactoring of the Digipede Agent.
If you aren’t familiar with it, NDepend is a static analysis tool that allows you to dive deeply into your code base. Its feature list is truly truly impressive. Here are a few things that stand out for me:
- The Code Query Language (CQL) allowing SQL-like queries of your code base, with a set of pre-canned queries targeting code quality, design, naming conventions, and much more.
- A visual tool with dependency graphs and matrices and more
- Ability to compare between different runs of the same project – at an incredible level.
- Of course, A command-line tool for incorporation into your build process.
I loaded up the Digipede Agent assemblies and – kind of like a kid in a candy store – I found myself heading off in 10 different directions at once. My thought process went kind of like this:
- Wow, look at these matrices and graphs, and all these CQL queries.
- This is really cool to have this level of information available with build integration!
- Wow, there sure are a lot of warnings here.
- Let’s fix them! Wait, let’s prioritize them and customize them and, etc.
(The experience reminds me of when I started using FxCop.)
I highly recommend this software . . .
- To help you make informed refactoring decisions; and
- to add design and code-quality criteria (and enforcement) into your build process.
So, why do I say this is a “must-have” tool? Because code quality is not a nice-to-have. Quality reduces maintenance and support costs and allows you to spend your time and money on more profitable endeavors.
Go buy it.
BTW: I would like to post some of the results I’ve gotten with the Digipede Agent, but I’m not ready to share that yet.
Tags: .NET, Digipede, NDepend
February 9, 2009 at 11:03 am · Filed under .NET, Grid Computing
I have been taking a closer look at IronPython for a prospective customer. Never being happy with “shoulds”, I am going to show how to distribute IronPython objects on the Digipede Network.
The first thing I did get our old Python sample running in IronPython. This was the first user-contributed sample (thanks to Sean True). That sample (see it here) uses Python COM libraries to invoke a job with the Digipede Network COM APIs. This didn’t submit objects, just executed a command-line application.
I’m happy to say that the code required very little modification to run under IronPython. The only difference is in the syntax of the “import” commands. Kudos to the IronPython team.
I’ll post the working code once I get a little farther.
Next step: distribute IronPython objects. Fairly straightforward, but I’ll write a reusable C# Executive to load the IronPython class definition.
Tags: .NET, Digipede, IronPython
January 9, 2009 at 9:07 am · Filed under .NET, Miscellaneous
A couple of years ago, I wrote Three deficiencies in the C# preprocessor. Those issues still exist, but a good thing about C# is you don’t get the following problem . . .
I’m working on a project for a customer that includes upgrading a VC++ 6.0 DLL to VC++ 2008. There are some good resources on MSDN for breaking changes (for a starting point, see http://msdn.microsoft.com/en-us/library/bb531344.aspx). Of course, I didn’t start there, I started by just trying it. Pretty quickly I came across some of the breaking changes related to STL (specifically that iterators cannot be treated as pointers).
But I also had a pretty bewildering problem. Hundreds of the following errors:
error C2371: 'std::tr1::placeholders::ks' : redefinition; different basic types
The source of the error was C:\Program Files\Microsoft Visual Studio 9.0\VC\include\xxbind1, line 298. Looking there, I found:
// PLACEHOLDER ARGUMENTS
namespace placeholders { // placeholders
static _Ph<_NARGS> _CLASS_NAME(_);
} // namespace placeholders
Doing some investigation I found that xxbind1 is related to the TR1 extensions to the Standard Library. The errors didn’t help me figure out where my code was including this file, so using the process of elimination (and the trusty binary search), I found that excluding the STL map header (i.e., <map>) solved this problem. Not an option, though, since the code in question relies on stl::map.
So, I tried several other avenues to figure out what I was doing wrong. While scouring all the application header files leading up to this include, I found the following line:
#define _ ks._
I thought, “wow, that looks awfully general for a preprocessor constant in the main internal header file”. Going back to the code in xxbind1, now that error makes sense.
Removed it, and everything is fine.
Arbitrary string-replacement before compilation is a powerful thing, but it can allow seemingly unrelated and unintended havoc.
Tags: .NET, c#, VS2008
October 27, 2008 at 11:42 am · Filed under .NET, Grid Computing, Web 2.0
Microsoft’s long awaited cloud platform has finally been unveiled here at PDC 2008. Late to the Internet, Microsoft hit it hard. Late to the cloud, Microsoft is doing the same with Windows Azure. Happily, this will put an end to all the guessing about what Zurich, Red Dog, biztalk.net, SSDS, Live Mesh, etc., actually are.
Of course, now begins the discussion of how all these pieces fit together.
This is not a simple approach like Amazon’s EC2 or Google App Engine. Not to trivialize either, but they are certainly easier to understand. Try explaining them to the proverbial grandmother — no problem, especially if you leave out virtualization and pythons 
(preemptive comment: I know AWS is much more than EC2 and that bigger and better things are coming from Google).
Regardless, the Microsoft Azure is multi-faceted. In typical Microsoft fashion, there is a lot for a developer to choose from:
- Azure Storage, Management, and Compute. Run WCF/ASP.NET based services, with work queues and data storage.
- Microsoft .NET Services, nee biztalk.net (wrote about here). This gives you an Internet Service Bus, Access Control, and Workflow Services. Messages and workflow in the cloud connecting other cloud and enterprise offerings. Very big deal.
- Microsoft SQL Services, nee SQL Server Data Services or SSDS. Eventually a relational model in the sky, currently not too different from Azure Storage.
- Live Services: Not too much detail on this today, but this is clearly what was “Live Mesh”: a rich synchronization framework, “live operating environment” for writing applications to across the Web and on user’s devices.
- Windows Live (Live Office, Live Sharepoint, Live Dynamics CRM, etc). In-cloud applications extensible by partners and users with in-cloud and in-premises solutions.
It all does fit together, and will be of immediate value to developers. As Marc Jacobs of Lab49 said to me afterward,
We could make use of all of these services today.
Damned straight. It is the openness of this platform, the ability of developers to mix and match the different components, and to do it between the cloud and in-premises solutions that makes this such a winner.
This last point is an important one. Microsoft is in a unique position to help enterprise IT bridge to the cloud. While I don’t think Amazon and Google will cede that market to Microsoft, their current offerings aren’t a natural fit.
Taking this all together — not forgetting Microsoft’s leading developer productivity story — it looks like a home run to me.
Tags: .NET, Azure, Microsoft, PDC, PDC2008, Reddog, Zurich
June 9, 2008 at 2:13 pm · Filed under .NET, Grid Computing
Last week Microsoft released the first CTP of the Microsoft Distributed Cache (code-named Velocity). 
I am definitely excited about this release. While Microsoft is not breaking new ground here, the addition of a distributed cache to .NET is a great addition to the platform. Certainly there are competing technologies, but Velocity will be a very simple choice for developers and ISVs because we’ll be able to count on its availability.
This ISV is interested, so we tried it out.
We have many customers who use our Executive pattern to load and cache job-specific data for compute-intensive jobs on the Digipede Network. These data are often fetched through WS calls or directly from SQL databases. Often this is performed in the Executive.Start method. Before Velocity, the code might look like this:
protected override void Start() {
// read the CBOData object from the database.
_cboData = ReadCboData(cache.Get(JobTemplate.Parameters["CBODataStore"].Value));
}
Including Velocity in this example is really easy. The following snippet adds use of the Velocity cache:
protected override void Start() {
// get cache
CacheFactory factory = new CacheFactory();
Cache cache = factory.GetCache("CBOCache");
// see if our CBOData object is already there
string key = JobTemplate.Parameters["CBODataKey"].Value;
_cboData = (CBOData)cache.Get(key);
// if not, read it from the database.
if (_cboData == null) {
_cboData = ReadCboData(cache.Get(JobTemplate.Parameters["CBODataStore"].Value));
// store it in the cache for later use
cache.Put(key, _cboData);
}
}
With a few lines of code, we reduce the load on the database server and network and spend more time computing. (I’m making an assumption with this simple code that all Executives don’t start at once, an assumption made obsolete by seeding the cache from a master application).
Of course, this is a simple example, but there are many other use cases. For example,:
- Digipede-enabled applications can share results;
- master applications can load the cache with job-specific data; and,
- others where baking Velocity deeply into the Digipede Network start looking pretty interesting.
I have seen many posts on “must-haves” for a Velocity RTM. I mostly agree with the lists I have seen. I’ll have a list too mostly from the ISV perspective.
Cool stuff.
Tags: .NET, Digipede, ISV, Velocity
June 6, 2008 at 8:53 am · Filed under Miscellaneous
I am routinely asked if the Digipede Network can run under Mono. The main scenario people want is to run Digipede Agents on a mix of Windows and Linux boxes.
My answer has always been the same:
Not now, maybe never, but this would not be technically challenging.
A while back we researched what it would take to get the Digipede Agent working on Linux. At the time, the Mono project was less mature. Finding out what worked was mostly a game of just trying it. We eventually gave up, because a dependency on WSE2 turned out to be a blocker.
A couple of weeks ago I decided to see what it would take to finish this task. I haven’t paid much attention to the Mono project over the years, but Moonlight gave me hope that Mono has made a lot of advances since we tried last.
It has. Still using Visual Studio, I #ifdef’ed out a few noncompliant functions and got it working with the Mono runtime. It was surprisingly easy. Kudos to Miguel de Icaza, Novell, and the Mono team.
So now I have it working in the lab on a Linux VM. Checking in to a Digipede Server, it is able to run our .NET development patterns. Of course, it can also run command-line applications with Linux-specific binaries. Pretty cool.
This is not ready for release. Not only is a supported product more than just bits, but I took a few shortcuts in getting the bits to work!
Now if someone asks me if it Digipede Network can run under Mono, I can say
It does in the lab, it may never get released, and this is ultimately a market and product decision.
So now we can do it, what shall we do with it?
Tags: .NET, Digipede, Mono, Novell
May 19, 2008 at 6:22 am · Filed under Miscellaneous
Dmitry Sotnikov provides the steps to get PowerShell — and of more general interest, .NET 2.0 – running on Windows Server 2008 Server core.
While this is a stopgap until Microsoft officially supports this configuration, it could still be useful.
Now the question for me is, will this work for running Digipede components?
Tags: .NET, Digipede, PowerShell, Server 2008, Server Core
May 9, 2008 at 6:18 am · Filed under Grid Computing
Dan has a write up of some of the enhancements added to this release here. He said we should have probably called it 3.0, but it is really more of a 2.5. We’ll be hosting webcasts soon going over the new features.
Thanks to the team for all the hard work in getting this out the door.
Follow http://twitter.com/010111011010111 for Digipede announcements.
Tags: .NET, Digipede, grid
April 28, 2008 at 1:00 pm · Filed under .NET
David Treadwell was on the latest Gillmor Gang talking about the recent Live Mesh announcement. David’s title is Corporate Vice President, Live Platform Services and has been described as Ray Ozzie’s point man on the Mesh.
It was a pleasure talking with him — and thanks to David for the LiveMesh invitations.
The synchronization capability in this preview is a big deal not in what it provides, but for what it promises.
That is why it is a little disappointing that there is such a heavy emphasis on Windows and Windows Mobile. I discount the coming Macintosh support because support for non-Windows mobile devices is really the issue. If iPhones and Blackberrys are out of the equation, then the synchronization story isn’t so compelling.
Nobody should be surprised about Microsoft promoting Windows. And I certainly am not, but Microsoft’s new openness had me hoping for a different alignment of Microsoft strategy. One in which their S+S play would de-couple the Windows, Office, Windows Mobile, and Live businesses. I saw this happening through the Silverlight runtime everywhere. I hoped that the mobile Live Mesh synchronization client would be written on top of Silverlight. I hoped that the next Office Mobile would be too. Then Live services could serve any device running Silverlight. And so on. I’ve written about this previously, so I’ll leave it at that.
Instead, Microsoft is approaching Live Mesh as a set of open protocols that anyone can implement. So, an iPhone version could be written by a 3rd party using the Apple SDK. Just implement the protocols — of which FeedSync seems to be the major part — and you are all set. That is very good and much better than requiring the use of a Microsoft runtime to make it happen.
But, in addition to the open protocols, I would still have preferred a vision where the Silverlight runtime lies underneath the Microsoft implementations of the Live Mesh client. That way, when the next big feature set for Live Mesh is released, the new client code could conceivably run everywhere.
I want to make one thing clear: I’m not saying that Silverlight in its current form could support this at all. And I know Silverlight’s (nee WPF/E) genesis emphasized presentation, but at the end of the day, it is a .NET runtime.
As David says (from the Gillmor Gang transcript on TechCrunch):
Treadwell: I really view mesh and Silverlight as orthogonal and complementary technologies. Essentially what the mesh client does, it’s the runtime for doing synchronization and collaboration those kinds of things. I view Silverlight as a runtime that does the presentation engine. Mesh doesn’t really have anything for presentation, Silverlight doesn’t really have anything for synchronization and mobile communications. Working together I think you have a very good thought there about the combination of these and how they’ll come together. We’re working actively on that but we don’t have all the I’s dotted and t’s crossed.
Yes. They are orthogonal if Microsoft says they are. And Live Mesh and Silverlight will somehow come together though this appears to mean in terms of presentation. Fair enough.
And more than a little cool.
Tags: .NET, FeedSync, GillmorGang, Live Mesh, Silverlight, Treadwell
Next entries »
I have been playing with IronPython a little. With the release of Digipede Network 2.2, I am now able to post the sample I wrote. It shows how to distribute IronPython objects on the Digipede Network. You can find it on the Digipede community site. See the posting there for details and download instructions.
About me
Main feed
Email subscriptions
