Compsoft Flexible Specialists

Compsoft plc

Compsoft Weblog Compsoft Website News Archive Privacy Policy Contact Us  

Saturday, November 15, 2008

Richard Thomason: TechEnd

The first great thing about TechEd is the buzz. You could spend a little time every month poking around on Microsoft sites, subscribe to their feeds and look at the key blogs. I don't do that and instead rely on the geek version of bush telegraph to alert me of newage that I need to be aware of. This works fine for major stuff, and the killer factor is, it's very low maintenance. At TechEd though, there's a load of people who've been beavering away at Microsoft things for a whole year and are busting to show you. You can spend days reading blogs and feeds and forums and really, it's all words; ten minutes in a TechEd presentation, and you can tell from the seniority of the speaker, the information content and the demo whether something is useful now or over the horizon, irrelevant or cool, forget it or must have.

Azure is a good example of this. I have a blx alarm that automatically goes off every time I hear the phrase "in the cloud", and until last week it has protected me from discovering much about it. I figured I'd better go to an overview since there was so much buzz, and ended up going to a couple more sessions to get the big picture and some practical demonstrations. If MS get the scalability / cost right, and if they make deployment a doddle, and if you can write your app in such a way that it is deployable in or out of the cloud without too much hassle, it could be big.

The second great thing is the random element. It's important to go to the widest selection of sessions that you can, since often the least likely turn out to be the most exciting. You can use the Robotics product to make a cut down version of Windows; that has to be good for something. You can also use it for loosely coupled but high performance applets which must inter-communicate. Virtual Earth, which I had pretty much written off because the Google offering is so good, is actually much better than I realised. There's no excuse now for customers to shy away from GIS functionality in their applications.

Finally, it's a great team builder. There's nothing like being in an away team to clear the air and refocus. It's good to be on the plane home too though. A week's uninterrupted geekery is enough for me.

Friday, November 14, 2008

Tim Jeanes: TechEd 2008 - Day 5

DAT317 - Project "Velocity": A First Look at Microsoft's Distributed Caching Framework

Velocity is Microsoft's upcoming caching framework. It's a distributed in-memory cache for all kinds of data - CLR objects, rows, XML, binary data, etc. - so long as you can serialize it, you can cache it.

This allows you to share cached information over a number of web servers, providing fast access to the data held in the cache. The cache itself is also spread over a number of machines, but you never need to know which server is holding your cached item. Velocity supports failover too, so if one of the cache hosts goes down, your cached item is still available from another host.

Objects are stored in the cache as named items, so the cache acts like a dictionary. Simple Get and Put commands can be used for basic access. Optimistic concurrecy can be used: Velocity tracks a version number of each item internally and only lets you update an item if there have been no other updates to it since it was retrieved. More pessimistically, you can use GetAndLock/ PutAndUnlock/ Unlock to prevent other threads or other servers updating items you're using. On top of the name of each item you cam apply tags, letting you retrieve sets of data with the same tag all in one go.

As you'd expect, you can configure the cache to specify how long it holds on to items before they expire and have to be retrieved from the backend database again. A nice feature is that it also supports notifications, so that if you update an item in the cache, that cache host notifies all others that the item has been changed. Of course this model has a bit more overhead, so which to use depends on how time-critical the accuracy of your data is.

Velocity also supports a SessionStoreProvider, making it easier to share session data over a number of web servers. This is about 200% faster than using SQL Server to store session state. Again this supports failover, so even if the primary machine that's storing your shopping cart goes up in smoke, that's not going to stop you spending money.


PDC309 - Designing applications for Windows Azure

This was a great session because it filled in a lot of the detail of things we'd learnt about Azure earlier this week, and because it gave some coding best practices for working on the cloud. The presenter has already deployed a number of real-life Azure-based applications, so it's great to be able to pick up what he's learnt the hard way.

I was a little surprised at some of the limitations of table storage. You can only have 255 properties per entry, which seems a bit arbitrarily small. Properties on table entities can only be the basic types (int, string, DateTime, Guid, etc.), as you'd expect, but there's a field size limit of 64K for strings and byte arrays. Each entity is limited to 1MB in total, though I suppose that makes sense - once your objects are getting that large they probably belong in a blob anyway. On the other hand, I'm glad to hear that they will be adding index support to table entity properties.

For now you only get indexes on the row key and the partition key (both of which are strings - up to 64KB each). As a pair together, they make the primary key for the table, which I think is perhaps a little awkward: surely the row key by itself should be enough? It would make cross-partition joins much easier, and we all love GUIDs anyway. However, I suppose this does push you in the right direction: cross-partition joins can potentially be more expensive (because Azure will always ensure each partition is held entirely on one physical machine), so I guess you should avoid them where possible.

As you don't get indexes on other properties (for the CTP, at least) this means you have to denormalise your data all over the place if you're going to want to do any kind of searching. More critically, as table storage has no support for transactions, it looks like we'll be waiting for SQL Data Services to come out before we try any real-life database-type storage in the cloud.

We also took quite an in-depth look at queues and how to process them in worker roles.

There is absolutely no limit to the number of queues you can have, so go crazy! One queue for each customer would be entirely feasible. The only restriction is the name of the queue: as queues can be accessed in a RESTful way, the name has to be something that you can use as part of a URI.

I quite like the way messages are handled as they are read from a queue. When a message is read, it is marked as invisible so that other worker roles don't pick it up. If the role that took it dies and never deletes the message, it becomes visible again after a configurable timeout period. The downside is that if a malformed message always causes an exception, it remains on the queue forever. the way to handle this is to check the time the message was created, when you read it from the queue. If it's been there for a long time, delete the message from the queue. What you do next depends on your application - you could log the problem, put the bad message onto a "bad messages" queue, or whatever.

It's possible (and more efficient) to read a number of messages from a queue at once, then process and delete them one at a time. Once the messages have been processed, you loop the worker role back to the start again - just by using while(true) - but sleep the Thread for a bit first if the queue was empty.

One point that was mentioned a few times was that you have to code to accommodate failure. Worker roles may die unexpectedly part way through processing a message, for example, leaving your table storage in a half-baked state. The fact that everything is stateless coupled with the lack of transaction support on the table storage means there's a whole bunch of craziness you'll have to code around.

The worker role has a property that returns its health status - you have to implement this yourself. An easy check is to see how long you've been processing the current message and flag yourself as unhealthy if it looks like you're stuck in an infinite loop. The health property does nothing in the current CTP release - in future Azure will monitor it to choose when to kill and restart the worker role.

One little gotcha: when DateTime.Now in the cloud, remember you don't know where your code is running, so you can assume nothing about what timezone you're in. Always use DateTime.UtcNow instead.


DVP315 - Do's and Don'ts in Silverlight 2 Applications

As we're just starting out in Silverlight 2, this was a really handy session. We got some tips from someone who's already made all our mistakes for us. Here they are:

Don't:

- Leave the default "Install Silverlight" message.

It's rubbish and if your user hasn't heard of Silverlight then they're not going to install it. Instead display an image that shows the user what sort of thing they'll see in the real application, with a message telling them what they have to do.

- Take a long time to load.

Instead load assets after the main app has loaded. Also cache assets in the isolated storage so they don't have to get it again next time. That way if there's no internet access, we can look in the local cache. It there, your app can work offline.

- Leave the default splash screen.

- Use width and height with

Resizing videos is very costly and wasteful in terms of CPU time, so show videos at their natural size only. Similarly, animating the size of text or complicated paths is also CPU intensive. To animate the size of text, build a vector from the text and scale that.

- Use a transparent background to the Silverlight app - it's very expensive. Admittedly it can look great though...

- Use opacity to hide something.

If you fade something out, remember to set Visibility=Visibility.Collapsed at the end.

Do:

- Show the designer mock data for use in Blend.

You can detect where a user control is being rendered by examining DesignerProperties.GetIsInDesignMode(this). Elsewhere you can check HtmlPage.IsEnabled.

- Use EnableFrameRateCounter=true when debugging.

This shows the current framerate in the status bar of the browser. Set the maximum framerate to something very high so you get an accurate indication of how your application's performing.

- Use BackgroundWorker or asynchronous calls.

The UI stops redrawing whilst code is running on the same thread.

- Use xperf to monitor performace.

This is a really handy tool that monitors and records the CPU usage of all processes. You can drill right down to individual procedure calls within your Silverlight app, showing you exactly what part of your application is being resource-hungry. Remember to run it as administrator.


WIN312 - Deep Dive into "Geneva" Next Generation Identity Server and Framework.

I'd heard references to Geneva and its claims-based identity model, so I thought I'd check it out in a bit more depth. At its most fundamental, claims-based identity supplies a series of flags that describe attributes of a user. Who provides the claims for a user can vary, but so long as your application gets the claims from a trusted source, you're good to go.

Under the Geneva framework, claims are passed around as security tokens, so are cryptographically protected by the claims issuer (normally Geneva Server). The Geneva framework takes care of all the cryptography for you.

The IClaimsIdentity, IClaimsPrincipal interfaces are available in .NET to code against, and are compatible with the existing roles-based security model. The Geneva framework converts information such as roles from Active Directory into claims.

The framework is extensible, so you can communicate with any type of issuer that you like (or that you invent), just so long as you have some way to convert their security token to a string.

You can add your own Claims Authentication Manager, where you can transform one set of claims to another. For example, given a claim that the user is in the UK and a claim that states they're over 18, you can emit a claim that they are legally allowed to buy alcohol.

This makes it really easy to define as many of your own application-specific claims as you like, and as Geneva already communicates with Active Directory, Live ID and Cardspace, quite a range of login routes are available right out of the box.

Geneva is a Windows component so is essentially free to use (or will be once it's fully live some time next year).

Richard Thomason: TechEd Day 5

17 The Future of C# - Mads Torgerson

Mads described the main areas of change in future versions of C#. It will be more declarative and functional, like LINQ and F#, and say what you want to do, and not necessarily how or when or in what order. These features can also help with asynchronous programming. Also more effort will be made in the area of concurrency as we address the need to make use of all those cores in our future machines.

C# in future will co-evolve with VB. Where in the past, features were put in as separate language features, now more effort will be made to ensure features in each language are equivalent where it makes sense to do so.

There's now a Dynamic Language Runtime which brings the compiler features to the runtime environment, and contains compiler features such as a parser and expression tree management, dynamic dispatch (useful for COM). In the language itself, there's a new dynamic type which delays type checking until runtime. Mads demoed this by pasting in a piece of JS code and converting it to C# with a few changes.

C# 4.0 will have optional and named parameters, nicked from VB. Mark parameters as optional by adding = value in the function declaration, and use paramname: value to pass named ones. Named parameters must be passed last, after the positional ones.

Improved interop with COM has been a big priority. The ref keyword is now optional as many parameters are passed by reference because it was more efficient at the time to pass a pointer. Obviously if it actually gets written to, you need to put it in. Parameters previously mapped to object are converted to dynamic, and so casting is no longer required.

There's some stuff on relaxing return types from functions - safe covariance and contravariance. If you're a compiler geek, see http://andersnoras.com/blogs/anoras/archive/2008/10/28/c-4-0-covariance-and-contra-variance.aspx for more details.

After looking at what the F# guys are up to, no doubt, and realising that in order to support LINQ they need a lot of the compiler at runtime anyway, they got around to implementing a compiler support library which gives you access to all parts of the compiler. I guess if you felt the need you could write your own IDE. Ho hum, this makes it a lot easier to do an Equinox method language which emits C#. Mads demonstrated an Immediate window in a few lines of code, and it gives you an Eval function for free. Of course Equinox has been doing all this for years... Hehe.

18 Debugging performance issues, memory issues and crashes in .Net applications

This was a repeat session, and was unfortunately full up, again. I'll have to catch the video of this one. Time for some work on Support's urgent 4.20 issue.

19 Universal sign in using Geneva - Sam Devasahayam and Sidd Shenoy

In a distributed, multi-company, cloud-based environment, it is no longer appropriate or even possible to use AD for authentication as in the past. To address this and other problems, the Geneva project provides a server based solution which uses industry standard token based security to provide authentication and other information services. For reasons which are not clear to me, pieces of information held by the system are called claims.

This technology does not require code, and can be set up via wizards in a very similar way to AD. It can be used as a simple alternative to merging directories when two companies merge - actually it looks like a no-brainer in that situation, and since Cardspace also has a Geneva version, it looks like internet user login is going that way too. It turns out that Geneva is the son (daughter?) of Active Directory Federated Services and Cardspace.

There are programmatic APIs similar to AD.

20 Distributed concurrent enterprise software using REST - Arvindra Sehmi

I've been meaning all week to take a look at the Microsoft Robotics Studio, and this is my chance, as this seminar explains how people are using it not only for the cool stuff like flying unmanned aircraft and factory automation but also for business uses where there's a need for a coordinated messaging system across loosely coupled nodes.

More information, this session has been removed, so I'll have to go to Windows Embedded CE and Robotics instead.

Windows Embedded Compact is the robot and phone ready version of Windows CE which also runs on personal navigation devices. It's not Windows Mobile 6.0. I'm confused, but hanging in there. It features a .Net runtime targeted for the devices required, typically about 10% of the size of the PC framework, and providing 30% of the functionality. WE Standard is a tunable version of XP Pro which can produce a small footprint OS suitable for embedded devices, and there's a fully featured enterprise version. There's also Windows .Net micro which is an embedded hardware access .Net.

A CE robot is a programmable system of sensor and actuators, so pretty much any automated system is a robot by this definition. Olivier configured and launched a build of CE (which takes about 20 minutes) then went through the requirements of a real time OS in excruciating detail.

CE is deterministically scheduled, inevitably, which runs counter to most of the low level software I've encountered in the past. However it does appear that they've addressed this and have a reasonably sophisticated mechanism for handling interrupts. Interrupt management is what embedded software is all about, so I'm not convinced by the architecture of this product. However it's a nice idea to be able to develop in a high level productivity environment. Olivier explained the work they've done to minimise the effects of uncontrolled things such as garbage collection.

The Concurrency and Coordination Runtime handles messaging and locking, so simplifies inter-taask management. Decentralised Software Services access individual pieces of hardware and are managegd by an Orchestrator process. This is quite similar to ordinary Windows, clearly.

Olivier put together a simple demo using a robot that he made out of a hoover, an interface card and a tiny pc with a wireless card.

Tristan Smith: Tech Ed 2008 Day 5

TLA311 The Future of C#

C# 1.0 gave us the ability to run managed code, C# 2.0 gave us generics, C# 3.0 gave us LINQ.
There's a strong drive towards dynamic languages and declarative programming at the moment. Declarative programming is about the what, not the how. So with LINQ, we say filter these results on Id, how it does that you're abstracted from.
Dynamic languages - simple and succinct, implicitly typed, meta-programming, no compilation
Static languages - robust, performant, intelligent tools, better scaling

C# 4.0 is enabling greater dynamic features.
Concurrency is a big challenge, we've hit a wall with clock speed so we're now going multi core processors. So in order to make use of these, you have to be able to have code run in parallel and asyncronously.

Microsoft are pushing a co-evolution with the languages that run on .NET so the features that c# has and VB doesn't (and vice versa) are being developed in parallel now, they should be much more the same.

Dynamic programming, DLR (Dynamic Language Runtime) allows a bridging between the static CLR and the dynamic world. Uses expression trees, dynamic dispatch, call site caching.
IronRuby, IronPython, C# and VB can all run against the DLR and allop interop between them.

Static type called dynamic allows dynamic calc - GetCalc(); int sum = calc.Invoke("Add", 10, 20) the actual type is deferred until runtime where the compiler will substitute the dynamic types for the actual types. The trade off is type safety but are more for situations where you don't have type safety anyway.

Javascript methods declared on the page can be called from the c# code by declaring an HtmlWindow.AsDynamic and calling window.MyJSMethod();
Moving Javascript to use dynamics in C# means you don't need to code in js, though you lose all intellisense because we don't have types to interrogate.

Optional and named parameters, currently we use a lot of method overloading OpenFile(string path, bool throwExceptionIfNotFound), OpenFile(string path) with optional parameters you can just use the first there with bool throw = true, setting a default if nothing is passed. Named parameters enable you to go OpenFile(throw: false).

Improvements with COM where before where there are optional parameters now rather than saying OpenFile("myfile.txt", Missing.Value, Missing.Value) because we don't have optional parameters currently where COM does, now we can just call OpenFile("myfile.txt").

Improvements also with co and contra-variance. Arrays are covariant, but not safely (can add a button into a string array of objects. Also you can now pass objects into methods based on base types.
This allows you to do this: public void ProcessCustomers(List stuff) and pass a List for example.

C# evaluator which is scoped so you can declare functions and use them in other evaluations. It allows you to do things such as declare and run code at runtime. Useful for things like plugins and even for secure remote debuggers.

PDC309 Designing applications for Windows Azure

We've seen a few more conceptual and technology sessions on Azure during the week but have yet to see any real life business use, that's what I hoped to get out of this and wasn't disappointed.

He started with what Azure is not, it's not a hosted windows server, or a sql server that you have access to. He gave things not to do and things you can't do.
He used the business example of the RNLI who monitor 10000 vessels and want to extend their safety of life services to the entire UK marine leisure market clearly to build in the redundancy would be a huge infrastructure headache. With Azure, they need only increase the number of running instances to match demand.

Working with queued items when you pull it off the queue, it hides it from other workers with a default timeout so if you don't finish working with an item on the queue (worker process fail for example) it'll get put back on for another worker to access.
Because of the parallel processing model, you need to write your code to expect failures at any point of inserting, deleting, processing etc in the case of a partial failure. Although not active in the CTP, in order to allow Azure to know whether to spin up another instance (in a non catastrophic failure scenario), you can override the health status. You need to forget ultimate accuracy though, because processes can fail after you're 80% though your app logic for example. Poisoned messages (things added to the queue that when processed cause the process to fail) need to be handled by us otherwise they'll keep being run.
The key message was that failure is guaranteed, deal with it. Compensate for errors, don't rely on state.

Useful information was that there are always at least 3 duplicates of your data for robustness.
Some best practices for scalability were shown with tables such as using the partition key as a point of scale, if you imagine every partition is stored on a separate server you can see how this scales well. Row key is there for performance with queries.

A really good talk that showed some really good best practices, much needed after the more abstract hello world examples we'd seen elsewhere.

DVP315 Dos and don'ts in Silverlight 2 Applications

Another useful best practice session. 75 minutes of what to be aware of, avoid and make sure you do.
For user experience, don't use the default Silverlight installer (a white image with a single "Install Silverlight" button in the middle) put your own in and point to it. This is a great place to show an image of what the user can expect when they've installed your app.
Don't load your application all at once, only load what you need to start the application, otherwise you make the user wait. You can download other resources to the user's isolated storage as a cache then load them next time they run it.
If they do have to wait, customise the wait screen by creating your own and pointing the onSourceDownload event at your progress bar update method.

There were a lot of performance tips:
Don't use transparent backgrounds, match the website colour. Don't animate text, make text into vectors and animate that, though don't animate paths if possible these are both expensive. Dispose of invisible controls, once you've animated to an opacity of 0, set vibility to collapsed if needed or otherwise remove them. They're never garbage collected until the references are removed. Use background workers and async webservice calls. Use GC.GetTotalMemory(true) to find out the current memory usage of your appliction. That way you can create a memory usage displaying control for debug. Use EnableRedrawRegions and EnableFrameRateCounter to give you a good view of how your interaction impacts performance. Don't set the width and height of media elements, provide alternate sized media, the media actually gets recoded, blended for transparency etc on resize.

Other tips were:
Hide controls that can't be interacted with such as textboxes in fullscreen mode where you don't have keyboard access.
Help designers out by outputting mock content for controls only shown if in debug mode using a method such as preprocessors (#if debug)

Neil Bostrom: Tech Ed 2008 - Day 5

Project "Velocity": A first look at Microsoft's Distributed Caching Framework

This session gave us a look at a very new project that has started at Microsoft. Distributed Caching is a common practice for any high traffic website. There have been a few products offering this on the market for years but this is Microsoft's first attempt to enter this market. We saw some demos of installing and running the cache server. The installation was simple enough and had some nice configuration options. The cache server had a very simple API for putting and getting cache items. The cache items can be split into regions, these regions restrict the item to being on the same cache server. This is helpful for grouping items together when you access them all at once. The down side of this is that it wont be distributed across multiple servers, reducing scalability. The project does look very promosing and would be ideal for any large scale websites we need to build. The final release date is due mid next year.

Real life experiences: Building your first service application

This session was presented by someone that had written an actual solution on the Microsoft Windows Azure Platform. As the cloud works very differently from anything we have worked with before, this was an excellent session to find out the pit falls and restrictions when working with the Azure Platform. The cloud has a few storage mechanisms, tables, blobs and queues. The session went into detail when working with the queues. When you take an item off the queue, it will just mark it has hidden, then once you have completed your work with the message you can remove it. If you do not remove the item from the queue (Like if your process died) then the message will reappear after a time out. This makes the system very resilient.

Do's and Don't in Silverlight 2 Applications

This session included excellent advice when working with Silverlight. The first item he covered was changing the loading screen for Silverlight. He then went into detail about avoid resizing videos and text as these have a heavy performance hit. He showed a nice user control for tracking memory usage in your silverlight application. Finally he covered Xperf, which is a nice performance monitor for windows.


Top 10 Web Mistakes And How To Avoid Them

This was the final session of the day. It was a nice interactive session on tips and tricks for building websites. Some nice tips on building websites, including headings to be searchable. Make your colour scheme friendly. The "what not to do" sites were shown, that were well amusing.

Thursday, November 13, 2008

Tim Jeanes: TechEd 2008 - Day 4

DAT318 - Microsoft SQL Server - Data Driven Applications from Device to Cloud

This was an extremely interesting session where we saw how SQL Server will evolve in the future, most particularly in connection with cloud computing.

Yesterday I was thinking that using SQL Data Services in the cloud would be pretty much identical to using SQL Server on a traditional server. It turns out that this is far from the case! Though they say they will support traditional database tables sometime soon, that's not the direction Microsoft are taking things and is not really the paradigm to use when thinking about cloud data storage.

SQL Services will store data in the Table Storage. It chucks everything into one big table, with all your different types of entities in together. It doesn't query the table using SQL statements, but rather uses LINQ. A typical statement might look like this:

from customer in entities.OfKind("Customer")
where customer["Surname"] == "Smith"
select customer

What makes it crazier is that there's no schema whatsoever to go with this table. If our application changes and we now need to add a MiddleName property to our customer class, we can just do that and start chucking our new form of customer object into the same table. We'll have to bear that in mind when we query the customers, of course. Even more so if you ever change the datatype of a property.

Another consideration is partitioning of data. Whenever you insert a record of any kind, you have to specify the partition that that record belongs to. The partition name is just a string and you can create as many as you want.

When you query data, your query can't cross the partition boundary: every query runs on a single partition. That's both a massive strength and a massive weakness. On the one hand, if you know which partition your data is in, your query will run very quickly. Let's say you partition on customer id (which was one option the speaker recommended), when you're considering data belonging to that customer you'll get results in no time. However, when you want to search for customers called Smith, you're stuck.

The only thing you can do to query across partitions is to launch queries in parallel, running one in each partition. In our customer search example, that means you're going to run one query for every single customer, which is clearly madness.

So what makes for a good partition decision? I don't know yet, and from talking to the speaker after the session, it doesn't seem anyone else really does either. For massive database (such as those for Windows Live Messenger or for Hotmail), a poor partitioning decision early in the project's lifecycle can have massive repercussions later on. Fortunately, Microsoft plan to have SQL Data Services make it a whole lot easier (for a start, possible) to change your partitions on a live database.

For the size databases we typically use at Compsoft, where (excluding binary data) they typically don't go far over 5GB, it's tempting to chuck everything into a single partition. That's not a scalable solution, though, and it's going to be a nightmare to go from one partition to multiple partitions later on. Also, the CTP release of SDS won't allow partitions bigger than 2GB. So should we arbitrarily assign each customer to one of a handful of partitions, then add more partitions as the application usage grows? Should we split on the first letter of the surname for now, moving to the first two letters when our application becomes too popular? Should we split on country? On the year the customer signed up? Which partition do we put our lookup data in (maybe it belongs in its own partition and we cache it all in the app)? Let's just try it and see!


WUX310 - Beauty and the Geeks: Developer-Designer Workflow

The Compsoft/BlueSulphur partnership is still a pretty new one, and we haven't really figured out quite how to balance the relationship between designers and developers. This session focussed on how to work side-by-side on the same Silverlight project. As this is something we're moving into, I hoped to pick up some pointers on good working practices. I wasn't disppointed!

We saw the development of a Silverlight control, what work the designer and the developer each did, and what considerations they had to make to how the other worked.

The project is available on codeplex: look for DeliciousSilverlight.

This is the first of seen of any real code behind in Silverlight development, and was impressed with how easy it all appears, and well-structured it is.

We saw some good practices for how to make your Silverlight controls unit testable: to do this we create a view model that exposes properties to the user interface. The designers can work against these in Blend, while the developers hook them up either to the actual view or to unit tests. We can detect whether the control is being rendered in a browser or in Blend, and by using dependency injection we can provide mock data that will appear in Blend for the designers to work against. This is genius!

When binding properties on the controls we develop to the view model, use the syntax: Text="{Binding MyTextProperty, Mode=TwoWay}" - this ensures the Text property of the control reads and writes to the MyTextProperty in the view model, both from Blend and from the browser.

We also saw how to make controls that are skinnable in Blend. Using the TemplateVisualState attribute on the control class exposes a number of states that the control can be in (visually). Then in code, you handle the various events that trigger different display states (mouse over, click, etc.) and use VisualStateManager.GoToState to change the control's appearance.

Using the DependencyProperty type, we can expose further properties of our custom controls to Blend. You use this to tell it which actual property in the class you want the designer to affect, set default values, and (optionally) set a change handler method to do anything you need to in order to reflect changes made in Blend on your control is real time.

That's pretty much all you need to do to enable the designers to use the Blend world they're used to to play with your controls and style them visually to their heart's content.


PDC307 - Microsoft Visual Studio 10: Web Development Futures

Here we covered some of the improvements that will be coming out in Visual Studio 10.

First up was the designer view for web pages. At last, this renders in a standards compliant way! Yay!

If like me you prefer to type your own markup in source view, it's just great to see that code snippets now work in there as well as in the code window. They're giving us about 200 out of the box as well as the ability to add your own. I think this will really make a massive difference in churning out easy pages.

Also, when it comes to deleting controls in the source view, triple-clicking an element now selects the whole of it, which will make the whole process a lot less fiddly.

There are some improvements to javascript intellisense. It used to work cross-file only, but is now available within the same script file. Also, if you have separate javascript documentation for intellisense purposes - such as the recently Microsoft-released jQuery support files - there's a fix that means those files (though included in your project) won't be shipped to the browser. This change is already available as a hotfix to VS2008.

Another cool new feature is config transforms. You can set up your base web.config file, then create transforms that describe (in XML) how that file should be modified for use in debug, release, staging, etc. You can create one of these for each Build Type you have defined. When using msbuild to perform your compilation, these can be called in using the switches /t:TransformWebConfig and /P:Configuration.

There are some improvements on the web site publishing front too. They've reworked the publish dialog box, and the publish tab in the project properties has been expanded to include such things as which files are included, which IIS settings, etc.


ARC309 - Using WPF for Good and not Evil

OK, it had been a long day of amazing stuff and my brain was pretty fried, so I was up for something a bit lighter, and (as ever) David Platt didn't disappoint.

We took a look at a few applications in WPF, where (with all the whizz-bang graphical power it gives you) it's extremely easy to make a truly bad user interface. It's also easy to make a really good one, but only if you know what you're doing.

The key fact is that most developers don't know what they're doing, and though a designer can make something look pretty awesome, that doesn't mean they're any good at designing a good user interface. The best user interface is one that the user doesn't notice. The effects available to you should be used sparingly, with subtle transitions appealing to the user's right brain to send subconscious signals. Any effect you use should have a reason for being used (and "because I can and it's cool" doesn't count).

I really must get round to reading The Design Of Everyday Things by Donald Norman.

Richard Thomason: TechEd Day 4

Today is a better day all round. My nose is no longer streaming from the cold I contracted on Monday morning, and because we had to get back on the Metro after the Uk developer party, I got a few minutes more sleep!

13 Writing a 3D game in 60 minutes - Dirk Primbs

I figured this session would either be mind melting or illuminating, but the 60 minutes angle was encouraging. The XNA game development environment has been well conceived, and gives a straightforward object oriented managed code interface to hardware accelerated DirectX game development for Windows and XBox 360. Furthermore you don't need expensive development hardware, just download the free Game Studio XNA development kit. There are starter kits to get you going; Dirk used elements of an Asteroids style interface as the basis for his game, and in about an hour implemented a ship, enemies, bullets, movement, collision detection and a simple background. Not bad.

The technical details are pretty much as you would expect for VS OO development in a 3D graphics environment.

Update method: repeatedly called to handle movement and other actions
Draw method: also repeatedly called.
Content class handles resources.
Meshes are collections of points.
Effects are rendering descriptions - lighting, position, shadow.
Shader is a compilable language which is sent to the graphics card.
Effects are matrices:
Projection describes the field of view.
View describes the direction of view.
World describes the position on the screen.
Moves are handled by Matrix.createTranslation.
Intersections are handled by BoundingSpheres (simple and effective).
Backgrounds are textures.

Etonyek: Green Computing through sharing - Pat Helland

I remembered this chap from last year as the one who annoyingly read out verbatim every single word on every single slide, and prepared to grit my teeth. This year he improved - only 90% read out.

His talk was aimed at providing a rationale for cloud computing that went beyond the computer industry, and focussed on power consumption, competition for resources, SLA's and possible solutions. Fortunately MS are opening a big datacenter late this year, so he was able to talk about that and avoid the strong impression that MS are 3-4 years behind Google and other players such as Amazon in this whole area; Google has been doing datacenters in anger since at least 2006, and the impressive Google File System was white papered in 2003: labs.google.com/papers/gfs-sosp2003.pdf.

But enough sour grapes already. PH said that in the US about 2% of total power goes on datacenters, which is more than on TV. Somewhat less is consumed in Europe (a Google effect?) but the same trend to increasing consumption is there; how can this be managed and possibly reduced? He described two papers on usage of common resources - the famous Tragedy of the Commons by Garret Hardin in 1968, and a development by Michael Heller in 1998 called the Tragedy of the Anti-Commons. Hardin noted that when resources are shared they are always over-used, with detrimental effects on both the resource and on usage; his example was shepherds grazing their sheep on common land - there is no incentive for shepherds to not graze or to have fewer sheep.

Heller observed that when ownership is fine-grained it is more expensive and difficult to share, for example land ownership and patents. These problems both occur when companies use datacenter and other computing resources. Mainframes were expensive, PC's were unscalable and had no redundancy, virtual machines provide some solutions, and Lo! The cloud provides further sharing opportunities. A variety of factors make shared virtual machines efficient; mixing application types on servers makes more efficient use of power and throughput.

PH went on to discuss SLA's and noted that large numbers of low utilisation VM's were more efficient than small numbers of high utilisation ones. Amazon, whose page requests typically contact 150 different services (holy shmoly) have extreme requirements for response times, and use percentage rather than average response times, as this mechanism delivers a far better user experience. Typically they require 99.9% of pages to be rendered in less than 400 ms.

14 Quick integration from Sharepoint to your application - Robert Bogue

As I arrived a little late after lunch, this presenter was demonstrating how to make SP go to an embedded link. Never mind, I thought, I'm sure we'll move rapidly on. In due course, after dealing with parameterised URI's, we moved on to how out of the box features such as single sign on were unusable, how Web Parts were a lot of work, how the demo in fact was not refreshing the application because the cache settings were incorrectly configured, how the Page Viewer Web Part creates many usability issues, how the Business Data Catalog creates an application definition that is difficult to edit manually unless you use the built in definition editor, how good the search SP facitilities facilities were and how all the details were on his blog and not in his demo. And at this point I had had enough.

I returned to the workdesk, plugged in and took a look at his blog - Robert Bogue [MVP]: http://www.thorprojects.com/blog/default.aspx. The lead article was currently Sharepoint Governance and Pandora's Box.

I guess I shouldn't say that this particular messenger should definitely be shot, as I might be accused of being uncharitable. If this wasn't a public facing blog I could be more specific, but instead I propose that some of the energy saved by Pat Helland's efforts earlier should be encapsulated and applied for his benefit.

15 Building advanced Virtual Earth applications - Johannes Kebeck

I was excited about this one - the Google API is a stonker and as VE is actually jolly good maybe MS have caught up or even overtaken it. Unfortunately Johannes had connectivity issues at the start of his talk which made the whole a little rushed, but he overcame the problem with a lot of energy, excellent knowledge and demos.

Virtual Earth (VE) has been going since 2001, and started as an investigation project on dealing with larger data sets, which later became the TerraServer project. In a 50th birthday speech, Bill outlined a need to be able to see the earth from his pc, including local weather, 3D images and even live content. 7 years and half a dozen acquisitions later, VE has most of these features in some form or other, including several sources of geodata, their own camera system and 9000 cores for obtaining and automatically processing 3D data, real-time weather, and rich and powerful manipulation APIs, as well as a lot of support functionality in SQL Server 2008.

In the near future when privacy issues are overcome, which basically amounts to processing out people and number plates, there will be street views as well as 3D cityscapes. Trees are added in by detecting them with IR, matching the type via a geographic library of trees and "planting" them in the places identified by the camera.

Johannes started with a bang and showed in Popfly an RSS feed from Reuters and VE on the same mash-up. However then he piped the output from the feed through a geoconverter service which gave him location information and displayed the result as pushpins in VE with popups containing the stories, no code required. This got a round of applause.

He them showed the Photosynth viewer at National Geographic, using the Sphinx, followed by a demo showing a floor plan of the MS building in Reading side by side with the building in Virtual Earth. He showed in a technology called MapCruncher how to link the two images together so that VE understands how to move when you move in the other window, a matter of a few trangulation points and scaling. This is easy to achieve because VE supports multiple layers (VETileLayer), which can include layers from VE and layers from your own data server.

VE can be integrated via JS with pretty much everything. BP has a fabulously named Hurricane Management System which uses MOSS to figure out where the weather is, which staff will be affected and so on. CRM has obvious requirements for sales staff, customer locations, service engineers etc.

He gave some details of the extensive geodata support in SQL Server 2008, including geometry and geography data types and advanced tesselating indexing functionality. He then did a demo showing data points sourced from his local machine in VE, and discussed how to display the right number of points on the map, optimising the work required to show the correct number of points quickly by choosing to evaluate on the client or on the server. For apps which have less than 1000 points, evaluate them all on the client as there is not much processsing overhead. However use an array in JS to pass the points to VE if there are more than a few. For < 1500 points use a ShapeArray. For more than that, consider using client or server side clustering.

There are a number of data import tools. A good free one is Martin Nielsen's SQL spatial tools. The best commercial one is Spatial ETL.

16 The Windows Vista Bridge: access Vista from managed code - Kate Gregory

This was a very straightforward discussion of how to call unmanaged code from managed. In addition, it's a way to call Vista features not avilable in the CLR.

Originally distributed in the SDK as a sample, Vista Bridge 1.3 is now published on Code Gallery, which is much easier to find than in the SDK samples.

Neil Bostrom: Tech Ed 2008 - Day 4

DAT318 Microsoft SQL Server: Data-Driven Applications from Device to Cloud

This session was about SQL Data Services in the cloud. Currently all the sessions on the cloud have been on the Windows Azure Platform which only has a basic storage service. With SQL Data Services it gives you more complex type hosting with support for queries and reporting. The SQL Data Services is a modified version of SQL 2005 running on Microsoft cloud. They are working on upgrading the service to run SQL 2008 under the hood to get all those nice features. All the data and queries are run via REST services so you can just access them through a browser if you want. They have built a nice LINQ provider to access the cloud services. The SQL Data Service play nicely with the sync framework giving you offline data in a local store if you need it. Currently the first version of the data services does not support tables (as we know them). The later versions will be coming with traditional table support once they work out the best way to manage them.

TLA316 CCR and DSS: A data-driven approach to distributed, concurrent software

I entered this session knowing nothing about CCR and DSS. Turns out its a really lightweight concurrency manager that allows you to run code concurrently leveraging multi core processors. The session had some nice demos on calling the CCR and how you would write the handlers code. I believe this technology could be helpful when we need to off load work to different threads in our asp.net pages, like emailing and sms (Nakheel).

PDC307 Microsoft Visual Studio 10: Web Development Futures

Having seen some sneak preview shots of Visual Studio 2010 in the Key Note speech, this session gave a full run down on the features coming out with 2010 relating to Web Development. A few key areas are being worked on, Html snippets, improved javacript intellisense and overhauled deployment features. The killer feature for us is to be able to have different web.config files for different enviroments. This is done by using transform on the existing web.config.

PDC305 Overview of .NET Framework 4.0

This session gave a flash tour of the new feature they are working on for .NET Framework 4.0. The main focus areas are Web, Client Apps and MEF. One goal with the new version is going to be to unify all the features between C# and VB.NET. That means that C# will be getting named parameters like VB and VB will be getting all the nice features of automatic properties and initialisers. The thing that was mentioned that put me off a little was that they will be removing the ref keyword requirement from c#. I liked this enforcement so its a shame they are removing it. Hopefully we will be able to turn it back on. On the web side, they are doing loads of work on AJAX to support loads more client side binding to reduce the number of server calls you need to make. Client side is getting lots of nice feature when working with WPF and WCF. One of the features that excited me the most was the preview of MEF, Microsoft Extensability Framework. This will be an awesome framework for loading and managing
plugins. This is something I really need for Tim the Enchanter as the plugin loader code in there doesn't work as well as I would want!

Tristan Smith: Tech Ed 2008 Day 4

DAT318 Microsoft SQL Server: Data-Driven Applications from Device to Cloud

The cloud version of SQL Server, has some strong focuses.
- To be able to securely access data on any device from anywhere.
- To do so with high reliability, availability and supportability.

It has automatic failover, by keeping multiple copies of the data, auto switching and spinning up replacement backups.
You can access SQL Services via HTTP, SOAP, REST, Atom using URIs.
There's no schema enforced as metadata is kept with the data so that sending your customer data isn't meaningless outside of the database. You can also return any content you're storing by URI, which for a blob returns the image content for that Id for example.

Microsoft Sync Framework, allows offline and cached mode capabilities for your application so you can reliably cope with occasionally connected scenarios. Sync Framework is being used as the lubricant in writing software plus services.

GEN01 Green computing through sharing reducing both cost and carbon

Pat Helland talked about how since 2001 the cost of hardware, infrastructure etc used in data centers has been outstripped by the cost of power.
He talked about how the current sharing models mean you end up with under utilisation of servers and over provision of service. If people were able to share their servers in a way that didn't mean someone would lose out, it'd make huge savings in power. Where individual ownership means you don't benefit from economies of scale, shared ownership means people always try to get 'Their share' which can lead to over utilisation.

The cloud computing model that Microsoft are using gives a solution to this, they can span the application use over multiple boxes so you get your turn at the all you can eat buffet without high cost and the other penalties of small use.

He showed a video of the Microsoft datacenter where they have sealed containers with tens of thousands of server boxes with only network, power and cooling going into them. This allows the economies of scale and reduced power usage scenarios he was talking about.

WUX310 Beauty and the Geeks: Developer-Designer Workflow

As we have a design team in house, this one was a good session to see how other people work with this dynamic.
The developer showed how by using a data and behaviour model disconnected from the design, the designer was able to work easily with existing tools.

States that you want to expose in Blend for the designer to design for can be specified in the code for the custom control. This allows the designer to just care about the 'SomeCrazyEvent' etc.
You can expose Dependency Properties to Blend by declaring them in the code so the designers can work against the same properties and events as the developers.

Using this model allows you to make a clean separation as well with the model able to take care of updating internal collections, updating properties and informing the Silverlight of the need to update itself.

They definitely gave us some better processes we can put into effect with our designer team. A valuable session.

ARC309 Using WPF for Good and not Evil

This was a really entertaining session, the guys had seen David Platt before earlier in the week and recommended his sessions. In this one he talked about how users really don't care about the pretty pretty stuff, not that they can't appreciate it when done well, just that you really have to think from their point of view. Typically (and unintentionally) interfaces are designed as you'd want to use them, the thing is, you are not your users.

The speaker showed how the sample applicaton for WPF is a good example of both shockingly bad and awesomely good interface design. It features a family tree which is shown as little people, a background vertical gradient dark at the top to light at the bottom has meaning. The darker it is, the further back in history it is, the people are outlines if dead. The coolness of WPF is that it's actually a treeview control (like the FileSystem view of Folders and files). In this instance, it's really well used as a treeview has parent, leaf and child nodes, just like a family has parents, siblings and children. By styling it up nicely, translating each node to a person in the tree, it has a really good user experience.
In the same breath the detail view has a horizontal gradient, dark to light which with text on it actually makes your eyes fight to focus on the text, it's enough to make you physically ill.

The speaker also talked about how the best applications are the ones you never notice, users don't actually want to spend time in your applications. The quicker you can get them to the things they want to do with your application and get them back to enjoying real life, the better. Pretty graphics are just sugar coating, nice, providing you get them out as quick as you can by making the application as usable as possible.

Tim Jeanes: TechEd 2008 - Day 3

PDC201 - A Lap Around Cloud Services

Wow! I'd heard a little about cloud computing and how it was the way forward, but I didn't expect it to be as different as this!

Windows Azure is the operating system that handles it, but it's not an OS you can buy and install. It's just what runs in the Microsoft cloud. You buy computing time and storage space (somehow - they haven't decided how to charge for it just yet) and work within that. To you it looks like you're running in Windows Server 2008 64 bit, fully patched with .NET 3.5 SP1 on IIS7. You don't have many priviledges there though - you certainly can't install anything.

The biggest change is how storage works. The file system's not at all what you're used to. Instead of files you have blobs, tables and queues.

Blobs are the most like files, and can be kept in containers, which are pretty much like folders. If you purchase only storage and no computing power, this is all you need to store all your files in the cloud.

Tables are nothing like SQL tables. They contain entities, which are collections of properties. You can mix types of entities within a table. Each entity needs a row key and a partition key. Row keys give you instant access to an entity; every other query results in a full table scan. You can partition your data into more managable chunks, and so if your query specifies the partition, only that partition will be scanned. If you don't know which partition your data is in, you can run as many asynchronous queries as you have partitions to speed the process up. This seems really weird to me - that there's no indexing at all - but we're assured that it's all designed to be scalable to the order of billions of rows.

Queues let you specify work items. You can buy large numbers of websites and worker processes, but as they can't talk to each other directly, they can only communicate via your storage. Queues are the way of handling this.

There's an SDK available at www.azure.com that lets you run a local mock of the cloud and develop against it. Once you're happy with your service, deployment is a doddle. You build your app, upload the compiled file, upload the Service Model file (that tells the cloud how many instances to run, etc.) and you're done.

Initially your service is deployed to a staging area for checking; deploying it there takes some time to get it uploaded and running, but swapping it with your live application is instant, thus ensuring you have zero downtime, even during upgrades.


DAT308 - Answering The Queries Your Users Really Want To Ask: Full-Text Search in SQL Server 2008

We were debating in the office the other day: when the user uses one of our search pages to search the customer table, and they type John Smith into the name search field, did they mean Name='John Smith', Name LIKE '%John Smith%', Name LIKE '%John%' AND Name LIKE '%Smith%', or Name LIKE '%John%' OR Name LIKE '%Smith%'. Well, Full-Text searching opens up a whole load of new possibilities.

Full-Text searching understands about word boundaries, it understands about different inflections on words (you can search for 'file' and get back 'filing'), and it even understands about thesauruses (you can search for 'customer' and get back 'client'). It can be configured to run on different languages. It can open documents (Word, PDF, etc.) that you're holding in binary fields in your database, pull the text out of those documents and search them too. If you hold documents of your own custom type, containing text in Klingon, you can make your own add-ons to cope with that too.

What's also nice is that the indexing all happens in the background - your searches will run quickly when you come to query the data.

We saw a lot of this at a previous TechEd, but immediately wrote it off as a bad job when we found out how broken it was in SQL Server 2005. The searching was done by the OS rather than by SQL Server, so if you searched on a text field and on an ID fields simultaeously, the OS would scan the text in every single record while SQL pulled out the one record with that ID, then compare the results and keep the intersection. Also, backing up the database didn't back up the Full Text Index, so restoring the database later would leave you with an index that was worse than useless.

Fortunately, this has all been fixed in SQL Server 2008: it does all its own searching, and the indexes are including in the backups. It all just works.

You get a rank for each full text search query as well, so if you say you want 'John NEAR Smith', you'll get every record containing both words, but the further apart they are, the lower the rank.

Still, I feel we're still a step away from idealism. There are so many options on how to search text data, it's fine if you're writing a bespoke SQL statement and understand what the user's actually looking for. Unfortunately what users want is a text box on a web page that acts like Google.

I guess we could run a few different full text search algorithms and merge the results, but merging ranks across queries is meaningless, unfortunately, so I guess we'll just have to pick one search algorithm and run with that. The jury's still out.


DAT307 - ADO.NET Entity Framework In Depth

Oh dear. What a disappointing session. We've been using LINQ to SQL for almost a year now, and were keen to see how we can upgrade to LINQ to Entities and what advantages it would give us.

Unfortunately, we only skimmed over the new features: the examples we saw pretty much may as well have been done using LINQ to SQL. In passing, they mentioned that the Entity Framework supports one entity spanning more than one database table, and polymorphism being allowed in the entity-to-table mapping, but no more detail than that, and certainly no demonstration. I knew this already: I wanted to know what this looks like and how to use it.

One new feature we saw in passing was this:
    from p in myContext.Products.Include("Category")
... ensuring that the Category object for each Product is built when the query is executed. Now that's a whole lot less restrictive than the LoadOptions we used in LINQ to SQL!

We saw a little more of use LINQ against ADO.NET Data Services that we saw on Monday: given a service that returns records, you can write LINQ against its result set - the clever thing being that the query gets put into the query string of the request, so the server only returns the records you want.

Still, if I'd spent the time on Google I think I would have learnt more, but bafflingly there's no wireless network availability in the seminar rooms this year. Sigh.


PDC209 - .NET Services Drilldown - Part 2 (Access Control and Workflow)

OK, to be honest this pretty much went over my head. I was tempted to cut and paste from Wikipedia and see if anyone noticed, but there wasn't enough material on there to make it convincing.

The key points, though, were that Microsoft are moving away from roles-based authentication and towards claims-based authentication. Their offering to help with this is currently called Geneva, which has a number of components to handle the various parts of the authentication process.

When securing a cloud application, the user calls tha app, which requests claims from a trusted claims provider. The claims can give details about the user's age, location, employment status, etc., which the cloud app can then map to its own application-specific claims. Those claims can then be checked at the appropriate points in the application to ensure that the user has the appropriate rights to perform various functions. Microsoft are working to the open standard for claims-based authentication, so the authentication can come from anywhere, provided the application trusts that claims source.


PDC210 - Developing and Deploying Your First Cloud Service

Here we saw a bit more detail about writing cloud services, going into a little more detail than this morning's session.

Configuring an application is a little different to the web.config file we're used to. Instead we have a ServiceDefinition file that tells Azure what your service looks like, and what settings you require; also you have a ServiceConfiguration file that gives the values of these settings for your particular installation.

In this session we also saw (briefly) how blob storage can be secured, and how you need to provide credentials for read or write access, and also how you can keep certain blobs public: public blobs can be accessed directly by URLs, and would typically be what you use for pictures you put in your website.

We saw how to code against table storage. You use ADO.NET Data Services to talk to table storage, which fully supports LINQ. You can make a DataContext for accessing your tables by inheriting it from TableStorageDataServiceContext - the constructor takes a StorageAccountInfo, which is all it needs to start accessing the tables. Your classes that describe the entities in the tables inherit from TableStorageEntity, forcing you to implement the necessary row key and partition key.

Though table storage is what's really used to store data in the cloud, I really hope it isn't the closest thing you get to proper SQL-like table access. I found myself wondering about the most efficient way to store our business entities in this framework, how to construct further tables to index them and make them searchable in a reasonable timeframe. However I pretty soon realised that someone else will do a much better job of it than I can, and, just like SQL Server does that for me in conventional data storage, SQL Services will do it for me in the cloud. Accessing it from our application may be a little different, but actually there's a chance it may be exactly the same.

Finally we saw a (somewhat rushed) example of writing Worker Roles. These are the only bits of executable code you get to put in the cloud other than Web Roles (i.e. the processes that run your web site or web services). These can't make any connections of their own to the outside world, and can only read data from your data storage area.

Each Worker you make has to run indefinitely - basically you deliberately trap it in an infinite loop (with suitable Thread.Sleep calls to ensure it doesn't gobble CPU (that you'll most likely have to pay for)).

The worker picks up jobs left for it in the queue (most likely by your Web Role) and does whatever you specify with them. I guess to me this is most like the scheduled clean-up jobs we have running on our web servers, combined with the long-running jobs our web apps sometimes spin up additional threads to execute.

All in all, cloud computing is the future whether we like it or not. Windows Azure looks to make it pretty manageable and shockingly reliable and fast. It's a bit of a jump in the learning curve, but not as big as it could have been. I feel sure that here at Compsoft we're extremely well positioned to start developing applications in the cloud and I just can't wait to get started!