From what I know of Time Machine you'll be able to roll back system files if you know which one you want. I don't know if you specify a date/time to restore to.
OS X really doesn't need a system restore. You can reinstall the operating system over the existing one, restoring all your system files to what was shipped. All your apps, settings and data remain on the machine. Windows isn't capable of that.
Regarding memory, OS X will use all the memory available to it by caching information so it doesn't need to go to disk as often. When it needs more memory it will remove some of the cache from memory. Even running one application OS X will gradually use up memory, that's just being efficient.
You might see a system slowdown if you let your hard drive get too full. OS X can't manage the data as well in that case. Try to keep 25% free. I've had system slow downs a couple times on my Power Book and it was a sign the hard drive was going bad both times. Might check that.
Will Leopard Provide Real Rollback?
Will Leopard Provide Real Rollback?