Expert Texture Home Contact me About Subscribe Digipede Connect on LinkedIn rwandering on Twitter rwandering on FriendFeed

rwandering.net

The blogged wandering of Robert W. Anderson

Dan’s Worst .NET Bug

Last Thurday, Dan posted Worst .NET Bug I’ve Ever Seen.  This post was the result of a Digipede customer support incident resulting from this .NET behavior.  We had some trouble tracking it down.  The unexpected exceptions were bad enough — and the deceptive exception message text made it worse. 

He posted code to reproduce the problem on a single thread (a tight loop of open, write, close).  The actual code in question included no looping, but multiple threads following a fairly common pattern (i.e., open temp file, write, close, delete, rename).  The code was properly synchronized, but it still threw exceptions.  To keep this simple, I’ll stick with Dan’s simple version:

while (true) { 
   using (Stream sw = File.Open(strFileName, FileMode.Create)) { 
         using (BinaryWriter bw = new BinaryWriter(sw)) { 
            BinaryFormatter bf = new BinaryFormatter(); 
            bf.Serialize(bw.BaseStream, this); 
         } 
   } 
}

Loop this code on one or more threads and you may eventually get a System.IO.IOException.  I say may because I can’t reproduce it, but Dan can and so can one of our customers. 

Given many caveats (e.g., the paths are valid, the proper permissions exist, the file is not opened elsewhere, etc.), this code should never throw an exception.  The unmanaged resources should be released when the Stream.Dispose method is called.  The file should be closed.  The types of exceptions that our customer (and Dan) got all follow from the fact that sometimes the file isn’t closed.  Add a Thread.Sleep(1000) after the using block and the problem goes away.

So, is this a .NET bug or a Windows bug, or just expected behavior?

The first thing I did was take Dan’s simple loop and write it using the Windows Platform SDK (i.e., using the Windows API, not .NET).  I left out the delete/rename part of the pattern to make it like Dan’s tight loop.  The code looks like this:

while (true) { 
    HANDLE hFile = CreateFile(fileName, GENERIC_READ | GENERIC_WRITE, 
        0, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);   

    if (hFile == INVALID_HANDLE_VALUE) { 
        printf("Could not open file (error %d)\n", GetLastError()); 
        break;; 
    }   

    byte buffer[1024]; 
    DWORD writtenBytes; 
    if (!WriteFile(hFile, buffer, sizeof(buffer), &writtenBytes, NULL)) { 
        printf("Could not write file (error %d)\n", GetLastError()); 
        break; 
    }   

    if (!CloseHandle(hFile)) { 
        printf("Could not close file (error %d)\n", GetLastError()); 
        break; 
    } 
}

This code works fine on Dan’s machine (i.e., never exits).  

Next steps

So, yes, this appears to be a problem in .NET.  My next steps are to . . .

  1. Check the .NET versions / are they somehow different between our machines?
  2. See if this is a known bug.  Connect.microsoft.com, here I come.
  3. Question my assumptions.  

To this last point, does .NET (or Windows) not guarantee that a file is closed when the last handle is released?  If this is somehow true . . . how much code have you written that assumes that it does?  Me, a lot.  OK, not a huge amount, but enough that I’m really surprised this hasn’t come up before.

I’ll follow up on this post when I get to the bottom of it.  I also have some complaints about the content of some of the exceptions.  In the meantime, any reader want to try to reproduce this?

Tags: , , , , , , ,

Reviewing the eWeek VS2005 Review

Peter Coffee just reviewed Visual Studio 2005 in eWeek: Visual Studio 2005: Bright Lights and Shadows.

This article seems less a review and more a warning to developers to think hard about adopting the Microsoft development tools. The article is not exactly anti-Microsoft. Mr. Coffee has a good point when he says that the developer gets great productivity gains at the cost of adopting the entire Microsoft strategy; however, is this any different from the other major integrated environments?

In a related interview on AttentionTech (see Coffee Talk), Mr. Coffee points out that in their labs, they cannot test across a wide enough range of scenarios to verify stability issues reported by the user community. As a result, he cannot answer questions regarding the stability of VS 2005 and the greater question: was it ready to be released? While I accept this as a limitation, I do wonder what it means for the validity of the review. How was time spent in the lab?

Refactoring versus 400Mb files
For example, the review mentions that VS 2005 finally supports refactoring. It is a real productivity benefit for developers and is important for people evaluating the platform. This topic gets just a paragraph. Does it work well? The review doesn’t say. My experience is that it does not (as I write this post, I am also manually propagating a refactoring change across 15 different VS2005 projects).

Iimmediately preceding the re-factoring paragraph are three paragraphs about why developers shouldn’t expect a single IDE to do everything for them. The shining example here is that VS 2005 cannot open a 400Mb text file. It turns out that none of the major IDEs support files of the size. Is this a more useful test for a review than seeing how well refactoring works? I don’t think so.

VB6
About 20% of the article is devoted to issues for VB6 developers. Isn’t this old news? I can completely understand why the VB6 community feels slighted by Microsoft; however, the greater good for the development platform built upon a common framework has been proven. VB6 developers were always treated as second class citizens in terms of both development tools and resulting capabilities. While this is still the case for VB6, VB.NET developers don’t face this hurdle. This could not have (reasonably) been done without changes to the VB6 language. VB developers are second-class citizens no longer.

Strategy
I recommend the interview AttentionTech interview (see Coffee Talk). He discusses his review as well as the greater Microsoft strategy regarding integration of VS 2005 and SQL Server touching on the future with .NET 3.0 and LINQ. He does a good job of explaining this strategy as well as the value of these products for developers.

Tags: , , , , , , , ,

Journey to .NET 2.0

After some discussion about the timing of a move to .NET 2.0/VS2005, we have decided to take the plunge and natively build for .NET 2.0 for (a currently planned) release in November.  The main reason we have delayed in this decision is that we need to continue supporting current and prospective customers who, for various reasons, are not ready to move from .NET 1.1.

 

There are a couple of things about our product that make this a bit more complicated than simply upgrading all of our VS2003 projects.:

 

1.      We offer a .NET API to our product.  Since we are committed to supporting .NET 1.1, we will have to continue maintaining a 1.1 version of the API’s main assembly.

2.      The Digipede Network can instantiate distributed objects, serialize them, and return them to the caller of the API.  If the caller is using the .NET 1.1 CLR then the serializing assembly cannot be running in a newer CLR; otherwise, deserialization errors will occur. To solve this problem we need to select the proper CLR based on characteristics of the caller (or, more practically, on configurable requirements).

 

These details are worked out (and basically functional), so what is left is integrating the new tools into development, build, and test.  We have already tested it all against .NET 2.0, so there is little concern there.  The CLR team has done a great job on compatibility.

 

Integrating the tools is (mostly) straightforward, but includes nant, NDoc, ResHacker, FxCop, Dotfuscator, NUnit, NCover, XHEO, Infragistics, NDepend, and Resharper. 

 

Some of these are critical for our commercial release and some can be dealt with later.

 

I’ll continue to post about this port with any gotchas I come up with.

Tags: , , ,

Changing CLR Version with ILMerge

Here is a trick that I found while looking at ways to handle .NET 2.0 while continuing to support .NET 1.1. The obvious way to support both is to build with VS2003 and use application configuration to direct the loader to prefer .NET 2.0 if available:

<?xml version="1.0"?>
<configuration>
  <startup>
    <supportedruntime version="v2.0.50727">
      </supportedruntime><supportedruntime version="v1.1.4322">
    </supportedruntime></startup>
</configuration>

That is OK, but what if you want to bind your assembly to .NET 2.0 and you don’t have the time to re-engineer your build and release processes?

OK, that is a lot of ifs, but it occurred to me that maybe ILMerge might work. ILMerge merges multiple assemblies into one, but what if it works with a single assembly? It turns out that it does:

  ilmerge input.dll /out:output.dll

creates a new assembly that is bound to the CLR that ILMerge is running. Of course, it is stripped of its strong naming key and signcode key; so you have to deal with this seperately.  The good thing is, though, that all of your resources (both managed and Win32) are in your resulting PE.

Of course, you can use ildasm and ilasm to do this too, but it takes more steps (or at least more attention to the arguments) to get the intended result.

Tags: , ,

FileInfo behavior change in .NET 2.0 Beta 2, fixed in RC

The other night we were performing tests of the Digipede Network on .NET 2.0 Beta 2 and came across an unexpected failure. The failing code instantiated a FileInfo object with the fileName argument of a nonexistent file. Then the code read the Attributes property. This resulted in the following exception / stack trace:

Unhandled Exception: 
  System.IO.FileNotFoundException: 
  Could not find file 'xxxxx'.
  File name: 'xxxxx'
    at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath
      at System.IO.FileSystemInfo.get_Attributes()

The code was supposed to read Attributes only if Exists was true. This was clearly a logic error, but my first question was: “How does this ever work?” As I didn’t have time to look into it further, I just went ahead and fixed the problem.

Today I tested this code again in .NET 1.1 and found that Attributes returns a default value (all bits on) even when the file cannot be found. I looked this up in the BreakingChanges.chm (that hasn’t been updated since May) and found no reference to this change in behavior.

I decided to test it again on .NET 2.0 Beta 2 (v2.0.50215) and confirmed again that I get the above exception. This appears to be an unintentional breaking change to Beta 2.

Then I tried it with the .NET 2.0 RC (v2.0.50727, the one from the Visual Studio 2005 RC that was handed out at ). This time it worked just like it did in .NET 1.1.

So, the problem came and went.

I figure that the Beta 2 behavior was reasonable; however, the CLR team apparently (and, I think, correctly) prefers compatibility.

Here is the code sample I used to test this out:

using System;
using System.IO;

namespace FileInfoTest {
  /// 
  /// Summary description for Class1.
  /// 
    internal class FileInfoTest {
    /// 
    /// The main entry point for the application.
    /// 
    [STAThread]
    private static void Main(string[] args) {
      // check arg length
      if (args.Length != 1) {
        Console.WriteLine("Usage: FileInfoTest file");
        return;
      }
      // get FileInfo
      FileInfo fileInfo = new FileInfo(args[0]);

      Console.WriteLine("Exists:\t{0}", fileInfo.Exists);
      Console.WriteLine("Directory:\t{0}", fileInfo.DirectoryName);
      Console.WriteLine("Name:\t{0}", fileInfo.Name);
      Console.WriteLine("Attributes:\t{0}", fileInfo.Attributes);
      Console.WriteLine("LastAccessTime:\t{0}", fileInfo.LastAccessTime);
      Console.WriteLine("LastWriteTime:\t{0}", fileInfo.LastWriteTime);
    }
  }
}

Tags: ,

Removing a hanging assembly reference

I don’t know if many people have run into this problem: a .NET assembly referencing another assembly in its manifest, though the referenced assembly isn’t used at all.

I don’t believe this is possible under normal conditions, but it is possible with an application that post-processes the assembly. Specifically, this can happen when using Dotfuscator 3.0. When obfuscating .NET 1.1 assemblies, it supports custom obfuscation attributes defined in a .NET library that they ship. Depending on the way you configure Dotfuscator, these attributes can be stripped from your obfuscated assembly. This is reasonable as the attributes have already served their purpose. There is a confirmed bug (that I’m sure they’ll fix soon) that leaves a reference in the manifest though all the attributes have been removed.

In most cases this doesn’t effect the use of your assembly. The CLR won’t try to load the referenced assembly (as there are no actual references to any types in that assembly). The one problem that I have found, though, is if your assembly is a library that others (e.g., your customers) will reference in their own projects. The assembly still functions properly, but undesirable warnings will be emitted by the compiler. And of course, this only applies if you are not shipping the unreferenced assembly.

As an interim solution, they (i.e., Preemptive support staff) suggest disassembling to il with ildasm, manually editing the il, and then reassembling with ilasm. Of course, this approach doesn’t work in an automated release environment. Understandably, they leave that as an exercise for the reader.

Here is what we did. Since all of our release builds are done with nant,
we made use of the built-in ildasm and ilasm tasks. Then we added a custom task that removes the reference from the il. The following code shows an example target assuming that the name of the library is ‘MyLibrary.dll’:

<?xml version="1.0" ?>
<project name="Hanging reference Example" default="rebuild" xmlns="http://nant.sf.net/release/0.85-rc3/nant.xsd">
    <target name="assembleobf">
        <!-- fix the Preemptive.ObfuscationAttributes.dll hanging reference bug -->
        <ildasm rebuild="true" output="MyLibrary.dll.il" rawexceptionhandling="true" quoteallnames="true"
            utf8="true" linenumbers="true" input="MyLibrary.dll" />
        <!-- define a task that takes an IL file as an input, removes the Preemptive.ObfuscationAttributes.dll, and
             writes the file out again -->
        <script language="C#">
            <imports>
                <import namespace="System" />
                <import namespace="System.IO" />
                <import namespace="System.Text" />
            </imports>
            <code>
            <![CDATA[
                [TaskName("removeRef")]
                public class RemoveRefTask : Task {
                    private string _obfuscatedLibrary;
                    [TaskAttribute("obfuscatedLibrary", Required=true)]
                    public string SomeProperty {
                        get { return _obfuscatedLibrary; }
                        set { _obfuscatedLibrary = value; }
                    }
                                                       
                    protected override void ExecuteTask() {
                        StreamReader inFile = new StreamReader(_obfuscatedLibrary + ".il");
                        string inContents = inFile.ReadToEnd();
                        inFile.Close();
                   
                        StringBuilder sb = new StringBuilder(inContents.Length);
                        StringReader stringReader = new StringReader(inContents);
                        string line;
                        while ((line = stringReader.ReadLine()) != null) {
                            if (line != @".assembly extern 'PreEmptive.ObfuscationAttributes'") {
                                sb.Append(line + '
');
                            } else {
                                for (int i=0; i < 4; i++) {
                                    stringReader.ReadLine();
                                }
                                sb.Append(stringReader.ReadToEnd());
                            }
                        }
                   
                        StreamWriter outFile = new StreamWriter(_obfuscatedLibrary + ".il");
                        outFile.Write(sb.ToString());
                        outFile.Close();
                    }
                }
            ]]>
            </code>
        </script>
        <removeRef obfuscatedLibrary="MyLibrary.dll" />
        <ilasm rebuild="true" output="MyLibrary.dll" resourcefile="MyLibrary.dll.res" target="dll">
            <sources>
                <include name="MyLibrary.dll.il"></include>
            </sources>
        </ilasm>
    </target>
</project>

They’ll fix it soon, I’m sure — but for now this will work just fine.

Tags: , ,