Do standard weakref cache for each native resource class (repeated code isn't really avoidable here sadly), seal these classes, private constructors so you have to use e.g Model.FromNative( IModel ) Garbage collect on Client LoopEnd, eligable Resource classes were never getting collected and leaking their native handles