Why is ASP.net on mono like a new pet that hasn't been house trained?

...because it leaks all over the place.

We've been working for a while now on an MVC (initially MVC2, now MVC3) application running on mono 2.10.2 with FluentNHibernate 1.2 sitting on NHibernate 3.1, built as a web app but also available via a kiosked browser on the server. This effectively means we've got a web browser pointed at the server 24/7. In addition, it's a fairly heavily AJAX-ed application, pinging for data every second or so, which means we've got a browser sending requests to the application every second, 24/7. Still, not exactly heavy usage.

We've had a very, very hard time getting this working stably, and all of our troubles seem to be coming from the mono stack (possibly nhibernate as well, but even with that problem, it looks more likely that mono is the root cause). If you're putting together a web application using mono, you should probably look out for the following:

Memory leaks when running with sessions enabled

This seems to be something to do with CacheItems being created at a phenomenal rate without (apparently) ever being garbage collected. Even if you see no reason whatsoever why your application would be generating CacheItems, something under the hood will be there merrily churning them out. This will, eventually, bring your system to its knees before the mono process is killed. Switching garbage collectors to sgen (newer, better, see below...) doesn't help. This is almost certainly why mono web applications mysteriously stop working under heavy load according to the mono project.

Workarounds:

 You can, as suggested in the above link, restart your mono process every so often to release the memory. Not an ideal solution, obviously. Alternatively, if you don't need sessions you can run sessionless, which seems to stop this leak from happening (so it seems the CacheItems are being generated by something in the session code...). Or you could figure out what the fricking bug is and fix it.

An apparent race hazard when starting up a web app

If you choose to accept that you'll need to restart mono every so often, the next thing to watch out for is an intermittent exception when starting up. This occasionally manifests as an object not set to an instance of an object exception, but most commonly will appear as duplicate key exception thrown by the RoleManager - even when you aren't using roles for anything. To be honest, this being a race hazard is a total guess, I have no idea what the problem is, but something which is so intermittent just smacks of a race hazard. Since we didn't need sessions enabled, we simply switched to sessionless mode and stopped pursuing this one. Interestingly, since addressing all of the other problems listed in this post we've not seen any of these problems. So perhaps again something to do with the session code in mono...?

Workarounds:

 No idea. This seems to have just gone away as we've worked around all the rest of the weirdnesses. Best of luck to you if you're trying to solve this one. Sorry I can't help more.

Running the MVC sessionless is broken in mono

You should be able to run sessionlessly simply by adding the tag

< sessionstate mode="Off" > 

inside the system.web node in your web.config file. Providing you're not actually using TempData for anything, this should be fine. However, you'll get the following exception:

System.ArgumentNullException: Argument cannot be null.
Parameter name: httpSessionState
  at System.Web.HttpSessionStateWrapper..ctor (System.Web.SessionState.HttpSessionState httpSessionState) [0x00000] in <filename unknown>:0 
  at System.Web.HttpContextWrapper.get_Session () [0x00000] in <filename unknown>:0 
  at System.Web.Mvc.SessionStateTempDataProvider.LoadTempData (System.Web.Mvc.ControllerContext controllerContext) [0x00000] in <filename unknown>:0 
...

Workarounds:

According to the mighty Stack Overflow this has been fixed in .net. Alas not in mono. We worked around this by creating a dummy temp data provider

public class DummyTempDataProvider : ITempDataProvider
  {
   IDictionary ITempDataProvider.LoadTempData (ControllerContext controllerContext)
   {
    return null;
   }
   
   
   void ITempDataProvider.SaveTempData (ControllerContext controllerContext, IDictionary values)
   {
   }
  }

And then making sure all controllers overrode the CreateTempDataProvider function (since all our controllers inherit from a common base, we just stuck it in there).

protected override ITempDataProvider CreateTempDataProvider ()
  {
   //return base.CreateTempDataProvider();
   return new DummyTempDataProvider();
  }

Using the nHibernate WebSessionContext leaks memory

Our application uses nHibernate for persistence. Unfortunately, for some reason we've not been able to identify, if you use the WebSessionContext (which is what you should be using for a web application), you'll find that you are, once again, leaking memory. 

Workarounds:

 We found that the ThreadStaticSessionContext has no such trouble and - on a hunch - tested out a hybrid session context we had lying around from an earlier project. This works fine, no leaks. The only real difference between this and the WebSessionContext seems to be that our HybridSessionContext uses the Items hashtable in the HttpContext directly to store the session whereas the WebSessionContext creates a new hashtable inside the items hashtable which uses the ISessionFactory as the key to the Session. Why this should cause a leak is a mystery; it may have nothing to do with the leak. But it's the only obvious difference.

internal class HybridSessionContext : CurrentSessionContext
 {
  [ThreadStatic]
  private static ISession threadSession = null;
  private const string webSession   = "YourNamespace.YourProject.WebContext";
  
  
  public HybridSessionContext(ISessionFactoryImplementor factory)
  {
  }
  
  
  protected override ISession Session
  {
   get 
   { 
    return (HttpContext.Current != null) ? HttpContext.Current.Items[webSession] as ISession : threadSession;  
   }
   
   set 
   { 
    if (HttpContext.Current != null)  HttpContext.Current.Items[webSession]  = value;
    else         threadSession        = value; 
   }
  }
 }

The boehm garbage collector causes further leaks

After all that, you may still be seeing leaks. We've found that the boehm garbage collector that mono uses by default is rather easily confused, especially, for some reason, when it comes to web applications. 

Workarounds:

Try switching to mono-sgen, the generational garbage collector (see the link above). We found this moved us from a still inexplicably-leaking application to one behaving normally... while testing on xsp. Because...

mod_mono causes even more leaks

After all of the above were resolved, and everything looked peachy testing locally through XSP, we pushed the system onto a target machine and ran through mod_mono instead.... and got yet more memory leaks.

Workarounds:

Run through xsp and use mod_proxy to serve it via apache. And you'll get back to not having leaks.

To summarise

Currently, I'd say that ASP.net on mono is a bit of a mess. You can work round the problems to some extent, but we've spent a ridiculous amount of time dealing with them. Unfortunately the mono mailing list on this subject seems to have gone very quiet (Dan in the list is the other part of the "we" that I refer to in this post), and we're sufficiently out of time that we simply cannot continue to bughunt for the underlying causes of these issues without at least some support from folks who know their way around the guts of mono better than we do.

svn switch --relocate for a git-svn repo

Today, to my great annoyance, I discovered that if you're using git locally hooked up to a remote subversion server and you find yourself in a position where, if using svn only, you'd want to do svn switch --relocate ( for example, if the server has its address changed), git-svn presents you with a world of pain. Since the svn url gets baked into all the commit logs, and these are used to keep your git repo synced with the svn server, you can't just edit the .git/config file and change the url as you'd do for a normal git remote.

I'm not going to claim any credit for figuring out what the hell was going on; after a brief period of free-form jazz swearing at the machine with twenty-odd suddenly disconnected git repositories, I found a handy post explaining how to resolve this bloody annoying state of affairs.

Since I had a lot of repos to switch, I thought it best to write a little bash script to sort it all out for me. Here it is so you, too, can ease the pain of the unexpected subversion repository renaming with less hassle. Or, alternatively, utterly hose your repo. Seriously, this rewrites your commit logs and could do all kinds of the nasty. At least back up your local repository first before using it. And even then, you use it at your own risk, etc, etc:

#!/bin/bash
# git-svn-switch
# by Justen Hyde based on this blog post:
#
# http://translate.org.za/blogs/wynand/en/content/changing-your-svn-repository-address-git-svn-setup
#
# Use at your own risk. For the love of cthulhu, back
# your repo up before letting this loose on it.

if [ $# -ne 1 ]
then
  echo "Usage: `basename $0` {new subversion url}"
  exit -1
fi

if [[ $1 = "--help" || $1 = "-h" ]]
then
  echo
  echo "Usage: `basename $0` {new subversion url}"
  echo 
  echo " Changes the url of the subversion repository a git-svn repo is connected to." 
  echo " Analogous to svn switch. Potentially a weapon of mass destruction. Use with care."
  echo " Run this from within your git repo. You only need one argument: the new url of the svn repo."
  echo " git-svn-switch will attempt to verify that the url is at least a svn repo before starting the switch"
  echo " but don't depend on it to stop you from doing summat daft."
  echo
  exit 1
fi

# get the current subversion url
SRC=`git svn info --url`
if [ -n "$SRC" ]
then 
  FROM=`echo $SRC | sed "s|/trunk||"`
  REPO=`svn info $1`
  echo "Checking $REPO is actually a subversion repository..."
  if [ -n "$REPO" ]
  then 
    echo "The new URL looks valid."
    echo "Rewriting the git history with the new url..."
    SED_FILTER="sed 's;git-svn-id: "$FROM";git-svn-id: "$1";g'"
    git gc
    git filter-branch --msg-filter "$SED_FILTER" $(cat .git/packed-refs | awk '// {print $2}' | grep -v 'pack-refs')
#Couple of pointless checkouts - on some repos the log changes seem to need flushing by an operation like this
    git checkout trunk
    git checkout master
    echo "Rebuild git-svn internals and updating the repo"
    rm -rf .git/svn 
    sed -i~ 's|'$FROM'|'$1'|g' .git/config
    git svn rebase
  else
    echo "Error: $1 Does not appear to be a subversion repository."
  fi
else
  echo "Error: This doesn't appear to be a git working directory, or it's a git repo that hasn't been created using a git-svn bridge"
fi

EDIT: I've added a couple of checkouts to the script which don't serve any obvious purpose, just before rebuilding the internals. This is because I've found on some repos that for some reason git doesn't seem to notice the logs have been rewritten (git log shows you logs with the old address, but attempt to re-run the rewrite and git will behave as if the logs have been rewritten). Subversion seems to get the old version of the logs in this case, and so any operations trying to use the subversion address in the logs fail. Doing these checkouts seems to flush the logs properly and allow git-svn to behave as expected. Weird. Unfortunately, this does mean you have to have a reasonably standard repository structure (i.e., one that actually has a trunk that can be checked out) or the script will fail.

Templates, inheritance and nested classes

I've recently had a world of fun and confusion with a strange combination of templating, inheritance and nested class in c++. The error messages along the way were rather weird, so in a bid to save time in future, here's the story:

Let's begin with a pretty standard templated class. It does useful things to some data which it keeps a hold of during its lifetime.

template <class T> class Foo
{
  public:
    Foo()  { /*...*/ }
    ~Foo() { /*...*/ }
    void SetStuff(T *data) { /*...*/ }
    void DoStuff(T *data) { /*...*/ }
    T *ViewStuff() { /*...*/ }
  private: 
    T *data;
}

OK, now let's assume that we have a set of helper functions that are only useful with the data in Foo while it's in Foo, and which should never be used externally but are always used whenever Foo wants to do stuff with the data. So we'll define a little nested helper class and squirrel the data away in it:

template <class T> class Foo
{
  public:
    Foo()  { /*...*/ }
    ~Foo() { /*...*/ }
    void SetStuff(T *data) { /*...*/ }
    void DoStuff(T *data)  { /*...*/ }
    T *ViewStuff() { /*...*/ }
  private:
    class DataWrapper
    {
      public:
        DataWrapper(T *data) { /*...*/ }
        ~DataWrapper() { /*...*/ }
        void HelperFunction() { /*...*/ }
        T *GetData() { /*...*/ }
      private:
        T *data;
    }
    DataWrapper *data;
}

Note that DataWrapper doesn't need to be templated; that's taken care of because it's already nested inside a templated class. Now let's imagine that SetStuff and DoStuff aren't safe methods to call all the time, and that sometimes you need to make sure that you can only use the safe methods, like viewing stuff. We just need a FooView. Fine, let's make a base class of Foo that has a copy constructor and only provides the safe methods. Then we can just take a FooView copy of Foo which we know is safe to hand over for use elsewhere. We want FooView to be the base of Foo, which is possibly slightly counterintuitive, because Foo does everything that FooView does and more besides.

template <class T> class FooView
{
  public:
    //Copy constructor to allow us to create views of a Foo
    FooView(FooView & source) { /*...*/ }
    ~FooView() { /*...*/ }
    T *ViewStuff() { /*...*/ }
  protected:
    //It doesn't make sense to create a FooView unless we give it something to 
    //copy. The descended Foo class will need a default constructor, though.
    FooView() { /*...*/ }
    class DataWrapper
    {
      public:
        DataWrapper(T *data) { /*...*/ }
        ~DataWrapper() { /*...*/ }
        void HelperFunction() { /*...*/ }
        T *GetData() { /*...*/ }
      private:
        T *data;
    }
    DataWrapper *data;

}
template <class T> class Foo : public FooView <T>
{
  public:
    Foo() { /*...*/ }
    ~Foo() { /*...*/ }
    void SetStuff(T *data) { /*...*/ }
    void DoStuff(T *data) { /*...*/ }
}

DataWrapper needs to be in the FooView class since both Foo and FooView need to understand it, but nothing else should (perhaps it's code that would be unsafe if used outside a FooView object). So far, so simple. But as soon as we try to implement anything in Foo that uses a DataWrapper, we end up in a world of pain.

Our first problem is that inside Foo, the compiler doesn't understand what a DataWrapper is. Let's add an implementation to SetStuff:

template <class T> class Foo : public FooView <T>
{
  public:
    Foo() { /*...*/ }
    ~Foo() { /*...*/ }
    void SetStuff(T * data)
    {
      DataWrapper *wrapper;
      wrapper = new DataWrapper(data);
      this->data = wrapper;
    }
    void DoStuff(T *data) { /*...*/ }
}

...which will result in the compiler error "DataWrapper was not declared in this scope". The issue is that we're in a templated class and the compiler doesn't know where to look for the definition of DataWrapper (which is also in a templated class). So let's give it a hint:

template <class T> class Foo : public FooView <T>
{
  public:
    Foo() { /*...*/ }
    ~Foo() { /*...*/ }
    void SetStuff(T *data)
    {
      FooView<T>::DataWrapper *wrapper;
      wrapper = new FooView<T>::DataWrapper(data);
      this->data = wrapper;
    }
    void DoStuff(T *data) { /*...*/ }
}

Which is pretty ugly. And leads to the baffling error "wrapper was not declared in this scope".

Huh?

But... but... that IS the declaration of wrapper!

Somehow the compiler has become so confused by the templating and whatnot that it's totally failed to parse

FooView<T>::DataWrapper *wrapper;

as a declaration. It's trying to perform some operation on wrapper as if it were already declared. I've no idea what. If you switch T for a type, the problem goes away -

FooView<double>::DataWrapper *wrapper;

works fine, apart from being a bit totally useless. So it seems that the compiler has missed the fact that

FooView<T>::DataWrapper

is a type name. Fortunately there's a keyword available to convince the compiler that no, honestly, it is a type name:

template <class T> class Foo : public FooView <T>
{
  public:
    Foo() { /*...*/ }
    ~Foo() { /*...*/ }
    void SetStuff(T *data)
    {
      typename FooView<T>::DataWrapper *wrapper;
      wrapper = new typename FooView<T>::DataWrapper(data);
      this->data = wrapper;
    }
    void DoStuff(T *data) { /*...*/ }
}

This is clearly absurd. If you're using DataWrappers in many places, your code is going to end up looking like it's been beaten to death with an ugly stick. Fortunately we can avoid all that nonsense by using a private typedef inside Foo:

template <class T> class Foo : public FooView <T>
{
  private:
    typedef typename FooView<T>::DataWrapper DataWrapper
  public:
    Foo() { /*...*/ }
    ~Foo() { /*...*/ }
    void SetStuff(T *data)
    {
      DataWrapper *wrapper;
      wrapper = new DataWrapper(data);
      this->data = wrapper;
    }
    void DoStuff(T *data) { /*...*/ }
}

And now you can just use DataWrapper inside Foo the same way as you would inside FooView - and as you probably expected to be able to use it in the first place. Obvious, eh?

EDIT - The typename keyword exists specifically to address this sort of situation, according to the description here. I guess it's one of those things that you either know (and hence will understand why you get an error about an undeclared variable from a line which you think is a declaration) or you don't and will be totally baffled by. It's not an error message that helps you towards an understanding of the source of the problem, that's for sure!