More WCF and IIS

So spent a few weeks playing with wcf 4 and both windows and iis hosting, i have a nice lite config pattern making some use of wcf 4's configurationless features and extending it all to make use of code defined settings.
I have to say i like the use of the new proxy and host changes, made getting a clean pattern together nice and easy (I have found the default bindings rarely provide sufficient control within an enterprise environment).
IIS osting for straight services works best with http bindings, WAS supports the use of TCP, and although more reliable than msmq, its still messy to get working properly (im not and have no intention of being an iis expert). and why use tcp over http really, you can fully configure binary transmission, and security in http, in addition to ssl and lets not forget IIS ARR routing (love shared config and ARR load balancing).
msmq is a bit different, played in iis and got it working. even put a service pattern in place that allowed the use of arr, but fiddly, messy and unreliable, which brings me on to a point...
MSMQ is all about safety, I see a lot of posts about how to get high performance out of msmq, and why, if you need performance use tcp or http services. msmq is about getting a message from a to b with high reliability, so who cares if it takes a couple of seconds longer (i will reject any code i review that doesn't use durable messages). Throttling is a great idea, allows you to control load easily, and prioritising is an anti pattern, there is a chance a low priority message will never get processed so why would you take the chance. So best bet is wcf with net msmq binding, hopsted as a windows service (your boxes should always be active passive at the very least). if your msmq architecture is overloading your boxes, split yp the services on to different boxes, reduce throttling caps otherwise your architecture is wrong.

Object Relational Mapper

I use orm's quite extensively (mainly nHibernate and Llblgen) at work, and had been thinking about writing my own lightweight orm for many many months, and made a few attempts, none of which got fully completed, until last April, when during a week off i finally sat down collected all my ideas got rid of the complicated one's and wrote it.
I eventually went with a pure explicit engine, i have played (original 2 or 3 versions) with implicit, but after spending a great deal of time with nHibernate, honestly i have come to dislike and distrust implicit engines. I would prefer to say when i want an update, and when i request data, go back to the database to get it. Database developers (Ms, oracle or whoever) have spent more time than i ever could in optimising the query engines, so why on earth do i think i can do better? answer i can't, additionally if the engine decides when and what to write to the db as it feels like it there is risk it will get it wrong, and whats the point of persistance with that kind of risk.
Now i have a small amount of implicity in that it supports cascade (too useful to ignor) but i have restricted it a little, firstly if saving (insert or update) it will never delete, (otehrwise you have to track what queries the objects relate to etc) and it will check the record in the database if it needs to. Delete requests also cascade, and do delete, but only dependancies (i.e. if foreign keys to objects your deleting and fk column is not nullable the record is deleted). Finally at the point of calling write requests i have a cascade tree you can define, so you can specify exactly how deep to cascade if at all.

So why post about it now? well although i have been very happy with it, and my query engine is ok, i think it could be better, starting playing with a fluent Linq Expression qurery engine, my initial findins suggest i should be able to parse the expression nicely to fit in with the core engine, and would be a much nicer .net front end, I'll post my results when i actually get round to it (i have this next week off so may start it in a couple of days).

Working WCF Peer Mesh Code

I've had a few requests for the working source code for the wcf peer mesh, those of you that have asked will be happy to hear i finally got around to digging out the code and cleaning it up a little, I have hosted on my new Open Source SVN Repository
http://svn2.xp-dev.com/svn/smpPeerMesh/trunk/PeerResolver1.0/
anyone who doesn't use svn can email me, and i'll zip up the files and send them out to you.
I've build this in VS 2010, but its .net 3.5 so should open in 2008 without any problems (again if problems give me a shout and i will rebuild in 08).
Hope its of some help, I have implemented this in a production environment to provide a wpf ui with live updates on a background worker thread.
To cover myself... I take no responsibilty for the use of this code or code written on the back of it.

IIS Wcf and msmq Load balancing proposal

So, the system I'm currently re-architecting is essentially a bunch of wcf (tcp and msmq) windows services sat on a single box and various types of client, for various reasons its approaching capacity.

We have been looking at various options to provide the magic, scalable available reliable platform. Some are obvious (and previously spoken about), and simply involve replacing certain heavy layers with simpler (designed for usee) push systems and efficient data access (don't get me started on loading the entire data set to get a customer name)...

Anyway, after a lot of research, looking at hardware and software options i think we have finally settled on an idea (at least until i have prototyped it all).

So couple of thoughts before i divulge. Firstly we only actually use netTcpBinding to provide simple binary data transport, which in fact is supported by http by specifying the binary transport in the binding. Http is significantly more flexible, and opens our options greatly. Tcp binding by definition doesn't strictly support per call load balancing, its a session transport protocol, and requires careful tweaking to provide something similar to good per call scaling. Finally after spending far too much time playing, IIS7 and Windows activation services for IIS hosted services is fiddly at best.

So http binding is the way to go, IIS support out of the box, and a lot of options opened up, so i started to look at IIS 7.5 and web farms. First thing i find is Shared Config and Content, store your content in a secure shared environment (obvious and supported in iis 6 with some playing). but the important bit is the shared config (there are some great videos out there demoing the tech, so get google-ing). Basically set your site/service up on your first (master) server. The goto Configuration Sharing (in iis manager) and export the config to a shared location (create a password as appropriate), then switch on shared config pointing at the export you created and check everything still work.
Now on to the interesting part... Go to your additional servers in the farm, in config sharing switch on sharing (pointing at your config again). IIS needs to restart, and on doing so you'll find all your sites / applications you set up on the master have been configured for you on the additional boxes.
Ok so that takes care of deployment, but what about load balancing, NLB doesn't really hack it for most uses, if your hammering the boxes, and cpu is high, you need to check your code and architecture (could be done better), its very easy and possible though to max out threads on a wcf service, especially if it has relatively long running processes, this is why we want a web farm, instance load balancing. Yes you can go for something like a Cisco ACE module for hardware balancing, but that has a big price tag attached to it. NLB isn't a true load balancer (and this bit is only my opinion) it hammers one box until that box is unhappy (very high load) before it moves over to the next in the cluster. In comes a much more sophisticated software solution from microsoft, Application Request Routing module for IIS (ARR). This only handle http url routing, hence http bindings for wcf, essentially you set up a new box (or active/passive failover cluster for safety). In iis manager install, then create your web farm, defining the incoming URL, and the target instances in the farm. Whats really impressive is the load balancing, you have some common options out of the box (such as round robin, weighted etc) or fully customised algorythms, the examples on the IIS site talk about geographic datacenter routing (ie you'll get routed to your nearest available datacenter).
All this true load balancing (thank you microsoft, i was starting to bang my head against a brick wall) is wonderful and all, but doesn't resolve our push requirements, i intend to rely heavily on MSMQ for this, (using durable messaging for improved safety). I've looked at nServiceBus and a couple of others, but to be honest its firstly another technology for me to guide my team through, and secondly i'm unsure how well proven it is at enterprise level. I don't really have particularily complex requirements, (its all in the high and mid level architecture) so I'm looking at remote routing, i've heard tell that this is only supported in the legacy MsmqIntegrationBinding, however it looks like an AD addressing issue which won't affect my project (You would have to try very very hard to convice me to use public queues with AD integration in MSMQ - its a nice idea but over complicates what is essentially - pass this message to that server/farm over there - i always know where i'm pointing to, or someone has designed something badly). So the plan is to host the queue on the routing server, and have service instances (IIS 7 with WAS msmq services under appfabric) pointing at the remote host. Now i have to be honest, not sure if this is going to work, i know msmq can handle remote hosts (with a small overhead during locking of a message for pickup - blocking other services picking up a message briefly), I just need to test this configuration (finger crossed).
Finally since i mentioned it already appfabric (specifically dublin components for now), this amongst many things provides improved was support, and importantly instance monitoring and logging (stunning configurable wcf logging can be used).
So i've gone on a lot, and nothing particularily concrete at this time, i will however post the results of my prototyping.

wcf and iis7

There are many good posts out there on how to create a wcf service, so not going to go in any detail, suffice to say that a service contract, its implementation (I like these in there own assemblies, as it makes referencing easier) and a wcf web site with the appropriate service.svc file pointing at your implementation. Then its all config config config. although .net 4 is apparently moving away from the config dependancy i've not had chance to play.

The easiest option by far is to host an http binding service, but there are performance drawbacks, so look at the project its expected use before deciding on this. if you need to pass lots of data or you need speed go with tcp, and although you can load balance net.tcp its easier with http.

net.tcp and net.msmq are a little more involved, first make sure you have installed the non-http wcf component of .net in windows components, then check the nettcpactivator etc services are running, if you have already installed .net 4 you'll have to remove before doing the above as it causes problems. once these are running though its safe to reinstall. Now add the appropriate bindings (with port for tcp eg 9191:* and machine name with queue for msmq). all thats left is to add the protocols. you'll see many posts with command promt lines to copy edit paste and run. i think its far easier to open inetmgr, right click on the parent site, select advanced settings, then manage site and where it says http add ",net.tcp" and/or ",net.msmq" as appropriate (no spaces) then do the same for the app containing your service under the site. I also like to use the same credentials for the app pool and net activator services to avoid any possible issues.

That should be it, the activator services are the listeners and activate the service call when a request hits the server.

One quick note on msmq, you have to create the queue yourself, and unless you really need to advertise the queue on ad (and generally you don't) keep the queue private, the only difference between public and private is that public uses actve directory, if you plan to point your client endpoints at a server name or ip, public is a waste of resources and will hurt performance for no good reason. I have a theory aout load balancing msmq services in iis, and it involves using a horizontal pattern (lots of services on one machine pointing at a single queue, which works very well) with a vertical infrastructure (many machines) by pointing the nodes at a single queue on the primary server which should in theory work. I have a collegue to thank for the idea, and will post soon when i have tested it.

Load Underestimations

WCF services in an N-Tier service based system is very useful, but is dependant on appropriate design. Im currently in the process of re-designing a processing system. On initial deployment and for 18 months after the system could cope with the load. but not any more, and I'm faced with a full architecture change.

So a few thoughts on designing scalable systems, 1 if you have heavy critical processes in the chain consider the use of msmq with throttling to manage load on these processes. The same can be said for external system dependancies, technology like msmq can help.

Don't poll production databases to look for specific data. This is obvious i know, but I see so many ideas flawed by this approach. Try instead to use a push system to handle such requirements.

Load balancing, ideally design your architecture to allow load balancing, obviously small systems don't require this, but enterprise level applications should support it, even if its not going to be used straight away, at least its an option. Using IIS to host the services is an obvious way to do this.

MSMQ host in windows 7

I've recently had a problem running a wcf msmq host in windows 7, specifically to debug a new service. Turns out this was purely down to the installation of msmq. Unlike Windows XP, Windows 7 includes msmq out of the box, therefore generally a fresh windows install is not on a company domain at this point, and thats the problem. To create and manage a public queue on a domain, the msmq service needs to be integrated into AD (just making sure the check box is ticked isn't enough).

So if you are having trouble, goto windows components, uninstall msmq, check your logged into the domain and install again. final step is to make sure that the application in question has permissions to manage msmq, in vs debug, just run as administrator, otherwise the service user (not tested this yet though) should be fine.