tag:blogger.com,1999:blog-67928498537001075402024-03-13T17:54:07.742-04:00ABC - Always Be Coding.NET development, economics and the Canadian consumer experienceJames McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.comBlogger38125tag:blogger.com,1999:blog-6792849853700107540.post-50685663251245735562017-07-09T10:14:00.001-04:002017-07-09T10:14:53.533-04:00Dependency Inversion and Microsoft Web TechnologiesWhen I started learning C# and ASP.NET about 10 or 15 years ago (!!), we rendered HTML with Web Forms and published services with ASMX Web Services. A little while later we moved on to WCF services.<br />
<br />
One of the limitations of those technologies was that the framework would only call the default, parameterless constructor of the page and service classes. To allow unit testing, I used "poor man's" dependency injection (DI), which I learned from Jean-Paul Boodhoo's videos on DnRTV. This is the approach where you have 2 constructors: a default one that calls the other one, which takes instances of all needed dependencies. The default constructor created instances of concrete classes that implemented interfaces. E.g.<br />
<br />
<script src="https://gist.github.com/claq2/03bfe46447642d3447755f1dce350544.js"></script>
I don't remember using, or even looking to mock frameworks. I think we created custom mocks by creating test classes that implement the interfaces and allowed customizing responses. I'm not sure which inversion of control (IoC) containers existed for C# then, or mock frameworks. If they existed, they were probably open source, which was frowned on at the bank at the time, even if just for testing. If they were allowed it involved some paperwork, anyway. (Banks like to have big companies they can call when they need support and who will be around a while.) Our pages and unit tests looked something like this:<br />
<br />
<script src="https://gist.github.com/claq2/42cc06d7d9ebe5e9bcc08e057342f3b9.js"></script>
This is fine if all you want to do is unit test. But what if you want to do integration testing, where you mock out the layer where you cross into other people's code e.g. the database or an external service? You'd have to do something like this:<br />
<br />
<script src="https://gist.github.com/claq2/efeb8fd2a1c93d42cc5dfe7173acabed.js"></script>
This wasn't something we tried to do at the time. We were a pretty inexperienced bunch. (Programming at a bank vs. programming at a software company is like practicing law at a bank vs. at a law firm.)<br />
<br />
If I'd have known more about the dependency inversion principle then, I would have been very skeptical about putting up with this limitation. I would have immediately gone searching for ways to insert something into the web request pipeline to control page and service creation. If I need to replace an implementation at the bottom of a dependency graph, it should be as easy as replacing one interface registration in an IoC container.<br />
<br />
Searching the web now it looks like it was possible, but not in an a way that made you feel good. They look like hacks, or the domain of .NET experts. If you wanted to stick to the SOLID principles, though, it's what you should have done.<br />
<br />
I do find it surprising that Microsoft gave us frameworks based primarily on object oriented languages (C#, VB.NET) that didn't let you observe object-oriented (OO) practices. Every example from that time involved creating instances of concrete classes in the page or service classes themselves. I even remember one example that told developers to drag and drop a new SqlConnection object onto every page. Not very maintainable. Perhaps Microsoft didn't think the existing MS web developers of the day could embrace these concepts, so many being ASP/VB6 devs.<br />
<br />
Whatever the reason, these limitations lead to some particularly gnarly code, completely lacking in abstractions and injection points. For example, the WCF service I'm currently tearing apart to allow mocking out the bottom-most layer. It goes something like this:<br />
<br />
<script src="https://gist.github.com/claq2/3c1d2a3061dfe554d1990cf3e6c5bfc7.js"></script>
The WCF class calls static methods. The first time one of them is called, it initializes a static instance, that is, a singleton, of an object that has instances of database and service classes. If you replace one of those instances during a test, you must set it back to null for reinitialization when you're done, or have every test set it before it runs. This is what you get when you ignore the SOLID principles.<br />
<br />
It wasn't until ASP.NET MVC 3 that Microsoft built IoC into ASP.NET MVC from the start, allowing controllers to take dependencies in constructors. Until then, people used IoC containers that implemented the complex looking code that allowed using DI in ASP.NET. I'm still surprised that it took so long for them to bake this in.<br />
<br />
The next time I look at a technology that is based primarily on an OO language, I'll be looking for the injection points, no matter how complex they are to use. If they don't exist, the technology will need a very compelling reason for me to use it.<br />
<br />James McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.com0tag:blogger.com,1999:blog-6792849853700107540.post-78935411931264483742017-02-18T12:25:00.002-05:002017-02-18T12:25:27.932-05:00WebAuthenticationBroker and OAuth2 UserCancel ErrorWindows Phone 8.1 programming can be a little... opaque. Yes, I was working on a Phone 8.1 project. My phone can't be upgraded to 10. I'm in the less than 1% of phone users.<br />
<br />
I wanted to build an app that can do OAuth2 authentication, so I started with an example from IdentityServer3.Samples on GitHub. It uses WebAuthenticationBroker to present the OAuth server's UI to the user. A client with an ID of "implicitclient" was missing from Clients.cs, but I just copied one of the JavaScript implicit flow clients. The sample worked - I could get both an ID token and an access token at the same time.<br />
<br />
I created a similar Client.cs in my existing project and pretty much copied the sample WinPhone example. I could get an ID token from my server and I could get an access token. But not both at the same time.<br />
<br />
The WebAuthenticationResult.ResponseStatus value was WebAuthenticationStatus.UserCancel, a very generic error that has many sources. The ResponseErrorDetail property had a more specific error number, 2148270093. I couldn't find many references to this number on the web, but in its hex form, 0x800C000D, I found results. It's a "URL moniker code" produced by IE meaning INET_E_UNKNOWN_PROTOCOL. The description is "The protocol is not known and no pluggable protocols have been entered that match." Still not much to go on.<br />
<br />
The callback URL for Phone 8.1 apps starts with ms-app://, which I thought maybe wasn't being recognized. I pointed my Phone app at the sample server and I could get both tokens at the same time.<br />
<br />
I debugged the Phone app and grabbed the URL from both my auth server and the sample one. The sample server's callback URL was quite a bit shorter. It started to dawn on me that Phone 8.1 uses IE11 and that it might have a fairly conservative URL size limit. It turns out it's 2083, which is not long enough to hold both of my server's tokens. My signing certificate's key is twice the size of the sample servers, making the token signature twice as long.<br />
<br />
So, how to shorten the URL?<br />
<br />
I was needlessly including some claims, so I cut them out. I read that elliptical curve keys are shorter, which makes for shorter signatures. IdentityServer3 doesn't have support for EC certificates out of the box, so that would have been some work.<br />
<br />
Then I finally stumbled across the idea of reference tokens. It turns out that they are the typical way to shorten an OAuth2 callback URL. Instead of the entire access token the URL contains a short identifier. Clients send the identifier to the server and the server looks it up from its database.<br />
<br />
After 3 or 4 days of beating my head against the wall, problem solved. Now I can login.James McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.com0tag:blogger.com,1999:blog-6792849853700107540.post-57919004130423263712016-12-14T23:18:00.001-05:002016-12-14T23:18:56.437-05:00Don't be Afraid of SingleOrDefault - Much Worse, Performance Problem EditionI found another example at work of someone not taking advantage of SingleOrDefault when making Linq-to-Sql calls, but in a much worse way. I previously mentioned the use of Any() and First() in <a href="https://jamesmclachlan.blogspot.ca/2014/05/dont-be-afraid-of-singleordefault.html" target="_blank">https://jamesmclachlan.blogspot.ca/2014/05/dont-be-afraid-of-singleordefault.html</a>.<br />
<br />
The new example constructs a query and calls Any() to test for the presence of at least 1 result. Any() is very efficient on its own because it uses SQL's EXISTS to just check for at least 1 record without reading anything about the record. This isn't a problem on its own.<br />
<br />
Unfortunately, this was followed by a call to ToList() and then [0] to get the first item, instead of First(). The effect of ToList()[0] is to run the query and pull every one of the matching records into memory and then take the first item. First() at least tells SQL to only return the TOP 1 item.<br />
<br />
Even worse is that because of faulty logic the code fails to add any query parameters, loading ALL of the records in the system of a particular type. Production has many tens of thousands of such records. Luckily, it only does this in a very specific case. If anyone has ever seen a problem they haven't reported it.<br />
<br />
So it's not enough to hunt down uses of First() and FirstOrDefault(). We also need to look for ToList()[0], or perhaps all uses of All().<br />
<br />
<br />James McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.com0tag:blogger.com,1999:blog-6792849853700107540.post-31005058481532488612016-10-14T14:03:00.000-04:002016-10-14T14:03:04.460-04:00Handling User Data Securely - Really, Really SecurelyImagine you're building a web app that requires a password from a user for encryption. This password is not the same as their login one, it's used only for encrypting and decrypting their data in the browser. The user enters the password, the app uses it to generate an encryption key that the browser can use and then encrypts and uploads data. Later, the user types the password in again so that they can work with the encrypted data.<br />
<br />
One of the things your app promises is that it never stores the password on the server in any form that another party could intercept or crack. And if someone compromises the user's computer or it gets stolen or seized the app promises that there is no trace of the password left behind.<br />
<br />
This means never including the password or a hash of it on any posts to the server, neither in form submissions nor AJAX calls nor in something like ASP.NET Webforms view state. Similarly, the app must not save the password in any form to a persistent store like the browser's local storage or one of the database systems that some browsers support.<br />
<br />
What options do you have for building such a system with a typical web system that involves form posts like ASP.NET MVC or WebForms?<br />
<br />
There are 3 ways a web app typically persists data that the user enters from one page to another:<br />
<ol>
<li>cookies written to disk by the browser and sent back to the server with the page request</li>
<li>a view state system that includes bits of data in hidden form fields</li>
<li>session storage in the browser to leave data behind for the resulting page to read</li>
</ol>
<div>
The first 2 obviously send data to the server, so they are not solutions. The third one does't send data to the server, and it might seem as good as being only in memory. However different browsers treat it differently. They don't all clear it when you close the browser, and if the browser dies or the power goes out it may be left on disk.</div>
<div>
<br /></div>
<div>
You might be tempted to encrypt the value before writing it to view state or session storage, but who controls that encryption key? If you do, then you have to manage it and may have to reveal it under court order. If you generate a key in the user's browser, then you have the same problem of storing something between page views.</div>
<div>
<br /></div>
<div>
You conclude that the you can only hold the user's password in memory, meaning a JavaScript variable. The problem is that they disappear every time you visit another page. The easiest solution to program is for the app to ask the user to enter their password on every page that needs it. Not exactly a delightful experience.</div>
<div>
<br /></div>
<div>
Obviously you want to minimize the number of times the user enters their password. One approach would be for the app to have a field that collects the password, perhaps at the top. The rest of the page performs AJAX-y form posts that submit encrypted data, receives HTML from the server and updates the DOM with the response. Something like WebForms' UpdatePanel or Ajax.BeginForm in MVC.</div>
<div>
<br /></div>
<div>
If the user navigates to another part of the app that doesn't need the password, though, they have to reenter it when they return. This might be an advantage because the app only asks for the password in applicable contexts. Users would learn to do everything they need to before navigating, or they'd use 2 browser windows or tabs.</div>
<br />
You could get the number of times the user needs to enter the password down to once per use session by using a single page app (SPA). If the app is all JavaScript and AJAX calls for data then it can keep the variable holding the password alive for the entire duration of use. If the user closes the window the password evaporates. (Well, it might still be sitting in RAM or on disk in a virtual memory file, but this seems beyond our control in a browser.)<br />
<br />
SPAs have an entirely different development process that you may have to learn, especially if you are used to server side web development (ask me how I know :). Putting your time in to learn will be worth it, though, if you want to give the user the best possible experience of entering a secure value exactly once.James McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.com0tag:blogger.com,1999:blog-6792849853700107540.post-68013296206308446422016-07-17T20:38:00.002-04:002016-07-17T20:38:49.966-04:00Code Learnability vs. Feature AddabilityI'd like to talk about made up words.<br />
<br />
No, really I'd like to talk about the tension between making a code base easy to learn vs. the ease of adding things to it, specifically about web applications.<br />
<br />
Imagine you had a slider control for your code that went from "easy to learn" to "easy to maintain".<br />
<br />
When you slide the control all the way over to "easy to learn", most of the classes disappear along with all of the interfaces and many if statements appear. Methods balloon in size and contain duplicate logic because there are so few classes to encapsulate it. It's very easy to trace data and logic through calls up and down the stack. Anyone reading the code will come across all of the decisions it makes. The reader won't have a good sense of why the code does what it does, though. The lack of little methods like DetermineRanking or classes like RankingEngine mean that such code lives all over the place.<br />
<br />
At "easy to learn", you can give a new developer your codebase and they'll be productive very quickly with little mentoring. Need to add a new data field when adding a customer? Just find the giant AddCustomer method or click button event handler and change line 654. A quick bit of functional testing shows that more related logic lives down on line 1233.<br />
<br />
Oops, the same logic appears in SaveCustomer and now you can add a customer with a new field but users can't update it. QA or, even worse, customer support opens a bug, new developer finds duplicate logic, copy, paste, fixed. Again and again.<br />
<br />
In some organizations, this is totally acceptable to devs, managers and customers. They can hire less skilled, cheaper developers and undercut the competition. Their customers are probably buying the software for some cost center in their business, so spending as little as possible is the goal. Everyone here operates at the low end of some market, which might make sense at the moment. As the dev shop or the customer grows they'll try to level up the software they make or use.<br />
<br />
(The field of Economics contains an idea that ideally every product exists at every quality and price point so that everyone can afford the version of a product that fits precisely with their income. If there were only shacks and mansions to live in you'd have to live in a shack until you can afford a mansion. Better to have a wide range of housing options, especially within a neighbourhood.)<br />
<br />
I'll bet the development shop's employee turnover is pretty high, though. As soon as a dev learns a little more about development they look for a more advanced employer. Again, some companies accept this and keep hiring from the ever growing pool of junior developers, which is fine.<br />
<br />
I'll also bet that successfully adding features takes weeks, including writing code and back and forth with QA. Such code is not unit testable, so people find all bugs during end to end manual or automated testing.<br />
<br />
At the other end of the slider, "easy to maintain", classes and interfaces abound, if statements largely disappear, but the structure obfuscates the code's flow. Frameworks and libraries, like inversion of control (IOC) containers and request pipeline handlers, automatically set up actions and dependencies. This means that for the app to know what to do with a class or method you only have to put a specific interface or attribute on it.<br />
<br />
The experienced developer's productivity goes way up and bugs go way down because of unit testability and the inherent design (e.g. code no longer allows the string "0" or "1" for values, it uses booleans). However, the major trade off is the learning curve and cognitive load of the code base. The concept count goes through the roof. With so many classes and seemingly magic behaviour, new developers need a lot of guidance with the design. That could be in person or in a recorded video or design docs.<br />
<br />
(I recently experienced this myself looking at code samples that rely exclusively on IOC containers to resolve dependencies. Some containers use an easy to follow "map interface IFoo to class Foo" syntax, while others use "map all of Foo's interfaces to Foo". The latter makes it harder to find the thing that maps IFoo to Foo with Visual Studio's Find All References because IFoo doesn't appear in the mapping. I even found one IOC container with a convention-based mapping method like "map all classes in assembly X to the interfaces matching their names". This meant that neither the classes nor the interfaces appear in the container setup code. Unless you find the conference video or series of blog posts where the author goes from original code to the heavily refactored version, you can end up puzzling over the approaches you see.)<br />
<br />
Adding features now takes days and involves mostly one way trips through QA.Production bug counts due to actual code problems go down.<br />
<br />
The code now requires more experienced developers who demand higher pay. This causes the software to cost more. Customers are happy to pay, though, because the software "just works", which they need to keep their profit centers working. Turnover drops.<br />
<br />
There is a joke (belief?) in programming that the more complicated and obscure the code the more job security you have because only you know how to update it. This depends on what makes the code complicated, though, and what the benefit is to the company. Are you using well documented but advanced libraries that do a lot of heavy lifting, or are you writing 1000 line methods of bit shifting code with single letter variables?<br />
<br />James McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.com0tag:blogger.com,1999:blog-6792849853700107540.post-32331533411477202532016-04-01T23:30:00.000-04:002016-04-01T23:30:07.211-04:00It Doesn't Matter How They Get The DataI was reading about using the SecureString class. This is a class that encrypts a string so that it is readable only on the current machine so that it does not hang around in plain text. Strings in C# are immutable, so they stick around until they are garbage collected, even if you set your reference to a string to String.Blank (""). The typical use is to collect the value from the user directly into a SecureString and shuttle it around on the current machine. As soon as it leaves the machine it must be turned back into a plain string, which is <a href="https://blogs.msdn.microsoft.com/fpintos/2009/06/12/how-to-properly-convert-securestring-to-string/" target="_blank">not a best practice</a>.<br />
<br />
I was looking into using SecureString in ASP.NET to handle a password. That password is passed in in plain text, and probably again when you actually use it, so there would be more than 0 instances in clear text in memory. Most StackOverflow answers point this out, but it would at least cut down on the number of instances as it is passed from method to method. Strings are a value type, meaning that every time you call another method with one as an argument .NET creates another copy.<br />
<br />
In addition, I kept coming across the same kind of comment:<br />
<br />
<ul>
<li>From <a href="http://stackoverflow.com/questions/4463821/is-there-any-benefit-to-using-securestring-in-asp-net" target="_blank">http://stackoverflow.com/questions/4463821/is-there-any-benefit-to-using-securestring-in-asp-net</a>: "Since the hypothetical attacker would be using a server memory dump"</li>
<li>From <a href="http://stackoverflow.com/questions/23775907/is-securestring-in-net-really-useful-for-web-application" target="_blank">http://stackoverflow.com/questions/23775907/is-securestring-in-net-really-useful-for-web-application</a>: "SecureString isn't useful against hackers who can read your application's memory. It only prevents accidental leaks via swap file, crash dump, etc"</li>
<li>From <a href="http://stackoverflow.com/questions/26190938/is-securestring-ever-practical-in-a-c-sharp-application" target="_blank">http://stackoverflow.com/questions/26190938/is-securestring-ever-practical-in-a-c-sharp-application</a>: "... it's unlikely for someone to gain this kind of access to your computer, but in the sense that if someone does gain this kind of access, the computer is considered (for all intents and purposes) compromised and I don't think there's any language or any technique you can use to defend against this completely" and "If an attacker already has the means for a heap inspection, then they most likely either (A) already have the means to read keystrokes, or (B) already physically have the machine"</li>
</ul>
<div>
Yeah, no one has ever been able to read a remote, web-accessible machine's memory. <a href="http://heartbleed.com/" target="_blank">Until they could</a>.</div>
<div>
<br /></div>
<div>
This reminded me of a comment a co-worker made about something you could do with a value from one of our databases: "How could anyone get it?"</div>
<div>
<br /></div>
<div>
It doesn't matter.</div>
<div>
<br /></div>
<div>
How did they get the Ashley Madison data? How did they get the Sony data? How did they get the U.S. Office of Personnel Management data?</div>
<div>
<br /></div>
<div>
It doesn't matter.</div>
<div>
<br /></div>
<div>
What matters is what more they can do with it. This is called "pivoting", or using one breach to hop a level deeper. Security through obscurity works until your whole database appears on the web. Any SQL system is one SQL injection vulnerability away from giving up everything. Any software involved in serving web requests is a buffer-overflow bug away from barfing memory.</div>
<div>
<br /></div>
<div>
Practice defense in depth to limit a small compromise from becoming much bigger.</div>
<br />
<br />
<br />James McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.com0tag:blogger.com,1999:blog-6792849853700107540.post-90490517260288157222015-03-13T23:42:00.000-04:002015-03-13T23:42:25.774-04:00Assert.DoesNotThrow Does not Preserve Stack TracesJust a short post about an issue with an NUnit feature.<br />
<br />
It has an assertion called DoesNotThrow that takes a delegate function as an argument. If the delegate throws an exception then NUnit will throw its own exception. The method is great at pointing out exactly where the code under test is. However, if an exception does occur, NUnit reports the exception type, but looses the original stack trace. There have been enough occurrences of lost stack traces that I've stopped using it. Now I just comment the actual test line and let the exception bubble up to the test runner.James McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.com0tag:blogger.com,1999:blog-6792849853700107540.post-70177228097405175802015-03-04T14:30:00.005-05:002015-03-04T14:30:59.222-05:00Make Classes not EnumsThe main C# code base I work with at work is full of generic objects and methods written to use enums to drive behaviour. I'm not sure of the reason. The people who created it have all since moved on, so I don't know who to ask. All I can do is live with the giant switch statements and undo them into classes when I can. Why classes? Let's take a look.<br />
<br />
The full example code base on which the gists are taken is available in <a href="https://github.com/claq2/MakeClassesNotEnums" target="_blank">https://github.com/claq2/MakeClassesNotEnums</a>.<br />
<br />
<h4>
The (Evil) Enums Way</h4>
<br />
Say you have a service that creates different kinds of backup jobs. One of its arguments is an enum that indicates what kind of job to create. It returns only 1 kind of object, Job, that is not abstract. This object represents all the different possible kinds of jobs. A property, JobType, indicates whether it's for backing up files or a SQL database.<br />
<br />
<script src="https://gist.github.com/claq2/beea9f8a70e4b280c63a.js"></script>
In this example, creating a job means calling a single CreateJob method that takes an enum argument that indicates the type. This might seem contrived, as in "who would build this", but it's easy for a UI to consume. Show the user a drop down with the job types and call the one method with the user's choice in the enum.<br />
<br />
In object oriented programming there is something called the Single Responsibility Principle. It states that a class should have only 1 reason to change. Another way to put it is that a class should have exactly 1 concern. The enum approach violates this in at least 2 ways.<br />
<br />
First, the Job class represents all possible job types. It needs properties for every job type. A change to any job type or the addition of a new one means changes to this class. Some properties apply to all types and some apply to only some types. How do upstream developers of the Job class know which properties to fill in for each type? They have to look at the documentation. I hope they don't miss one of the comments.<br />
<br />
<script src="https://gist.github.com/claq2/faa8f9088b288ebbde36.js"></script>
Second, the JobService has 3 concerns - determining which type of job to build, verifying the values and mapping input values to properties. If any of these aspects change this class must be updated. The more you need to change it, the more risk there is that you will introduce a bug. Especially if the switch has many clauses with pages of complex logic.<br />
<br />
The cyclomatic complexity of this method is already 7, meaning you need at least 7 unit tests to completely cover it. This approach will only get harder to test fully and understand. There's already a little extra logic for Sql2014Jobs. How long before the logic becomes so tied together that you have to write tortured logic to get exactly what you want?<br />
<br />
Think about what load and save methods in the JobService class will look like. Nearly the same - more switches with extra if statements for the special cases.<br />
<br />
<h4>
The (Righteous) Classes Way</h4>
<div>
<br /></div>
<div>
There are 2 sources of cyclometric complexity in a system - the conditions that the business defines and those created by developers. You usually can't reduce the business's conditions, but you should be able to control the developer-created ones. The way you do this in an object oriented system is usually with classes and inheritance.</div>
<div>
<br /></div>
<div>
In this example one might wonder why the interface doesn't already know what kind of job it need to create in response to the user's actions. Well, it does. Outside of a text-menu driven console app, there shouldn't be any logic needed to decide what to create. The user chose to click on a button or use a drop down to pick the job type. The UI now has enough information to call a job-specific method and expect a job-specific result. Hence, separate job service classes.</div>
<div>
<br /></div>
<script src="https://gist.github.com/claq2/d960f6be9cda9ee7d760.js"></script><br />
<div>
<br /></div>
<div>
No conditional logic. The logic for SQL 2014 jobs is contained in a class that only has to do with SQL 2014 jobs. Any change to File jobs does not involve anything related to any other job or the services. As written, the services don't really add any value to consumers. They could just use the job constructors themselves. Imagine they had load and save methods, though. Those methods will be specific to the job types.</div>
<div>
<br /></div>
<div>
The logic for constructing the SQL 2014 job's instance name and for rejecting non-root path file job is now in those classes. There is no chance that a change to file job logic could impact other job types.<br />
<br />
<h4>
A Hybrid Approach</h4>
</div>
<div>
<br /></div>
<div>
Let's say you have a system that looks like the first example that has a single method and for non-technical reasons you cannot easily change the interface. Are you stuck? No.</div>
<div>
<br /></div>
<div>
Change the service to be or use a factory, as in the <a href="http://en.wikipedia.org/wiki/Factory_method_pattern" target="_blank">factory pattern</a>. Job-specific logic is still in separate classes and the service now has just 1 responsibility - to invoke the right service and return the result.</div>
<br />
<script src="https://gist.github.com/claq2/fd0e98bf735115728e3a.js"></script>
It is a lot more code than the original enum-only version, but it more closely follows the SOLID principles that make software more maintainable. The other classes are the same as in the class version, except the AbstractJob class. It now has an abstract ToJob method that each specific job class must implement to convert itself into a generic Job object. Again, this logic is encapsulated inside each specific class.
<br />
<br />
<h4>
Summary
</h4>
<br />
Enums and switches make for less code, but over time it becomes harder to understand and more easily broken. In an object-oriented language classes are there to segregate logic and responsibility for better maintenance. It's a little more code, but you will introduce less bugs.James McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.com0tag:blogger.com,1999:blog-6792849853700107540.post-17111074763601024712015-01-21T14:36:00.001-05:002015-01-21T14:36:48.113-05:00Handling Exceptions: Everything You Ever Wanted To KnowExceptions have been around for a long time, but some seemingly experienced people still struggle with them. I will explain how to handle and communicate them in various .NET applications.<br />
<br />
All exceptions in .NET inherit from System.Exception. You can't throw an instance of just any class; it must inherit from Exception. So, catching Exception means that nothing will get past your catch clause.<br />
<br />
<script src="https://gist.github.com/claq2/270ed6c7808d48486770.js"></script>
If there is a specific type of exception that you know you can handle, you must catch it before more general exception types. The class inheritance structure indicates which exceptions are more specific. ArgumentNullException inherits from ArgumentException, which inherits from Exception. So, if you want to handle the null exception differently from the argument exception, you must catch the null exception first. Each exception will be caught only once and by the first catch block that matches.<br />
<script src="https://gist.github.com/claq2/cde9437f46e875d9ed96.js"></script>
<br />
You can also add a "finally" clause after your catch clauses, in which you run code that must be run whether there was an error or not.<br />
<br />
<script src="https://gist.github.com/claq2/eeaf2ec1700330a7b8eb.js"></script>
In the above examples and the following ones you'll see a variable called ex that holds the Exception object. You don't have to declare such a variable if you don't need a reference to it. Also, omitting the exception is the same as catch (Exception).<br />
<br />
<script src="https://gist.github.com/claq2/846373ce1157ed2147ed.js"></script>
<br />
<h3>
Default Exception Handling</h3>
<br />
Before we discuss how and when to handle exceptions, let's first look at how .NET handles them if our code doesn't.<br />
<br />
Here is the code for a set of apps that throw exceptions in different ways: <a href="https://github.com/claq2/Exceptions" target="_blank">https://github.com/claq2/Exceptions</a><br />
<br />
<h4>
Console</h4>
<div>
<br /></div>
<div>
Any exception in a console app, whether on the main thread, a new Thread/ThreadPool thread or started with the await keyword produces this:</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-yUtOtqCJf5Y/VHSNwIBUNRI/AAAAAAAAANM/6yZ9Gq_Dqdc/s1600/Console.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://3.bp.blogspot.com/-yUtOtqCJf5Y/VHSNwIBUNRI/AAAAAAAAANM/6yZ9Gq_Dqdc/s1600/Console.png" /></a></div>
<div>
<br /></div>
<div>
Very straightforward, no ambiguous choices for the end user.</div>
<h4>
</h4>
<h4>
WinForms</h4>
<div>
<br /></div>
<div>
When a WinForm application generates an exception on the UI thread that the code doesn't handle (e.g. in a button click event handler), .NET shows the user this:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-GfoPi8qkrDU/VHSGZOa40xI/AAAAAAAAAMc/oVI5xz8hzJ8/s1600/WinForm.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://2.bp.blogspot.com/-GfoPi8qkrDU/VHSGZOa40xI/AAAAAAAAAMc/oVI5xz8hzJ8/s1600/WinForm.png" /></a></div>
<div>
<br /></div>
<div>
What's a user to do? In my opinion, this is too much information for a non-developer to deal with, and the user has no idea what each choice's implications are. Do not rely on this dialog for end users. We'll see how to avoid this later.</div>
<div>
<br /></div>
<div>
If you start a thread, either using new Thread or ThreadPool.QueueUserWorkItem, and it generates an uncaught exception, .NET will terminate the application:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-VlQA5oOcKpo/VHSITmUV8eI/AAAAAAAAAMo/GMw5sqaIllE/s1600/WinFormBackgroundThread.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://2.bp.blogspot.com/-VlQA5oOcKpo/VHSITmUV8eI/AAAAAAAAAMo/GMw5sqaIllE/s1600/WinFormBackgroundThread.png" /></a></div>
<div>
<br /></div>
<div>
I have a Debug option here because Windows knows I have a debugger installed (Visual Studio). Again, we'll see how to prevent this later. The runtime will log this in the Application Event log along with the stack trace.</div>
<div>
<br /></div>
<div>
An exception thrown on a thread started with await results in the same prompt as the exception on the UI thread.</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-C8j8kf1joZo/VHSJPEwPthI/AAAAAAAAAMw/6T9Zk1IxnWk/s1600/WinFormAsync.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://4.bp.blogspot.com/-C8j8kf1joZo/VHSJPEwPthI/AAAAAAAAAMw/6T9Zk1IxnWk/s1600/WinFormAsync.png" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
When you let .NET's manage your threads one thing it gives you is much better exception handling. Though again, the user must make an uninformed choice. The runtime doesn't add anything to the event logs because it handled the exception.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<h4>
</h4>
<h4>
WebForms</h4>
<div>
<br /></div>
<div>
An exception on the thread that is executing a web page request (e.g. in Page_Load or a button click event) shows:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-4MVmqMOhDyI/VHSKN4RrfDI/AAAAAAAAAM4/BXj3hHd75bA/s1600/RemoteWebForms.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://4.bp.blogspot.com/-4MVmqMOhDyI/VHSKN4RrfDI/AAAAAAAAAM4/BXj3hHd75bA/s1600/RemoteWebForms.png" /></a></div>
<div>
Also known as "the yellow page of death". Only this request has come to an end. The system is still serving other requests. By default, when the request originates outside of the web server there is no stack trace giving away potentially sensitive information about the app. When viewed on the web server it will show more information. There is no event log entry because the runtime handled the exception.</div>
<div>
<br /></div>
<div>
The same thing happens when an uncaught exception is thrown on a thread started with await. Again, .NET handles it gracefully.</div>
<div>
<br /></div>
<div>
A truly insidious thing happens when a ThreadPool thread or one started with new Thread throws an unhandled exception: the .NET worker process dies. The symptom might be invisible to the user that caused it, but every request that w3wp.exe process was serving will come to an end. If the app uses in-process session state, those sessions will be lost. If the app uses Windows authentication, all of the auth sessions are gone and users will need to re-authenticate. (Ask me how I know! :) We'll see how to guard against this later. The runtime will record the exception along with the stack trace in the Application event log.</div>
<div>
<br />
By default, the settings for an Application Pool (the thing the runs your ASP.NET website AKA the worker process w3wp.exe) will disable the pool if something kills the pool 5 times in 5 minutes. This is called Rapid-Fail Protection. When this kicks in, IIS returns HTTP 503, Service Unavailable for subsequent requests. If you find yourself having to disable this feature then I hope it's a short term mitigation while you hunt for the thing that's throwing exceptions on background threads.<br />
<br />
<h4>
</h4>
<h4>
WCF</h4>
</div>
<div>
<br /></div>
<div>
An exception thrown on the main thread or an async thread sends an HTTP 500 error with this XML:</div>
<div>
<br /></div>
<code>
</code>
<br />
<div>
<div>
<code><s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"></code></div>
<div>
<code><span class="Apple-tab-span" style="white-space: pre;"> </span><s:Body></code></div>
<div>
<code><span class="Apple-tab-span" style="white-space: pre;"> </span><s:Fault></code></div>
<div>
<code><span class="Apple-tab-span" style="white-space: pre;"> </span><faultcode xmlns:a="http://schemas.microsoft.com/net/2005/12/windowscommunicationfoundation/dispatcher">a:InternalServiceFault</faultcode></code></div>
<div>
<code><span class="Apple-tab-span" style="white-space: pre;"> </span><faultstring xml:lang="en-US">The server was unable to process the request due to an internal error. For more information about the error, either turn on IncludeExceptionDetailInFaults (either from ServiceBehaviorAttribute or from the &lt;serviceDebug&gt; configuration behavior) on the server in order to send the exception information back to the client, or turn on tracing as per the Microsoft .NET Framework SDK documentation and inspect the server trace logs.</faultstring></code></div>
<div>
<code><span class="Apple-tab-span" style="white-space: pre;"> </span></s:Fault></code></div>
<div>
<code><span class="Apple-tab-span" style="white-space: pre;"> </span></s:Body></code></div>
</div>
<code>
</code>
<br />
<div>
<code></s:Envelope></code><br />
<code><br /></code>
<code><br /></code></div>
<code>
</code>The WCF proxy that Visual Studio/wsdl.exe generates throws a System.ServiceModel.FaultException. As the text above indicates, you can have the server send the server code's stack trace by changing a web.config setting. This might be acceptable in a development environment, but you don't want your service leaking implementation details in production.<br />
<br />
Exceptions on background threads kill your ASP.NET worker process, w3wp.exe, just like WebForms. It's the same engine and threading mechanism. Again, the runtime records it in the Application event log.<br />
<br />
<h4>
</h4>
<h4>
MVC</h4>
<div>
<br /></div>
<div>
Like WebForms, both exceptions on the thread processing a request and on a thread started with await result in an error page:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-8d6NFl_VNRo/VHSmjLVSYTI/AAAAAAAAANc/v4THyqGPq7M/s1600/RemoteMVC.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://4.bp.blogspot.com/-8d6NFl_VNRo/VHSmjLVSYTI/AAAAAAAAANc/v4THyqGPq7M/s1600/RemoteMVC.png" /></a></div>
<div>
And like WebForms, an exception on a background thread (new Thread, ThreadPool) kills the w3wp.exe process serving the request, with all the same effects. This includes an Application event log.<br />
<br /></div>
<h4>
</h4>
<h4>
WebAPI and HttpClient</h4>
<div>
<br /></div>
<div>
WebAPI is very webby. Exceptions on the main thread and async threads that are not wrapped up into an HttpError and HttpResponseException result in an HTTP 500 Internal Server error and the following JSON:</div>
<div>
<br /></div>
<div>
<code>
{"Message":"An error has occurred."}</code></div>
<br />
You can have WebAPI send more information by setting config.IncludeErrorDetailPolicy = IncludeErrorDetailPolicy.Always in WebApiConfig.cs's Register method. With this enabled, an exception not thrown as an HttpResponseException will send:<br />
<br />
{"Message":"An error has occurred.","ExceptionMessage":"Oh noooo!","ExceptionType":"System.Exception","StackTrace":" at MVC.Controllers.ExceptionController.Get() in c:\\Users\\xyz\\Documents\\GitHub\\Exceptions\\MVC\\Controllers\\ExceptionController.cs:line 19\r\n at ..."}<br />
<br />
Oddly enough, wrapping up a caught exception in an HttpError and throwing HttpResponseException will cause the stack trace to be null.<br />
<br />
There are 2 ways to detect an error with HttpClient: examine HttpResponseMessage.IsSuccessMessage or call HttpResponseMessage.EnsureSuccessStatusCode to get an HttpRequestException. EnsureSuccessStatusCode will not deserialize the above message. It's up to you to turn it into an object after determining that IsSuccessMessage is false. Here I've built my own class in a console application that has most of the same properties as HttpError, because HttpError lives in System.Web.Http.dll, which is part of the WebApi.Core package. I also needed a custom exception type so that I could specify the stack trace:<br />
<br />
<script src="https://gist.github.com/claq2/48e464467917a4f76c9a.js"></script>
I understand that HttpClient is not specifically for interacting with WebAPI, but I am a little surprised that nothing natively converts WebAPI error responses into exceptions on the client side. ServiceStack built handy client and service components that share an error message format. This allows the client to reconstruct exceptions for you.<br />
<br />
<h4>
</h4>
<h4>
WPF</h4>
<div>
<br /></div>
<div>
WPF applications have a consistent approach that I like. In all instances the application dies with this message:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-CtSOeEAuQjw/VHSnCu-QqqI/AAAAAAAAANk/nAwtOY20ogQ/s1600/Wpf.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://4.bp.blogspot.com/-CtSOeEAuQjw/VHSnCu-QqqI/AAAAAAAAANk/nAwtOY20ogQ/s1600/Wpf.png" height="162" width="320" /></a></div>
<div>
<br /></div>
<div>
Like a console application, there is no choice for the user. I like this because the state of the application is unambiguous.<br />
<br /></div>
<h3>
</h3>
<h3>
Handling Exceptions</h3>
<div>
<br /></div>
<div>
The general rule to catching exceptions is to catch only ones about which you can do something. For example, catch a timeout exception so that you can retry on behalf of the user. Or ignore an exception when deleting something that isn't there.<br />
<br />
There are a few situations where you want to catch all exceptions:<br />
<br />
<ol>
<li>In background threads. As mentioned above, any unhandled exception in a background thread will kill your process, including your ASP.NET worker process.</li>
<li>When you want to translate one application layer's exceptions or .NET exceptions into another layer's exception type.</li>
<li>You want to throw a different exception in order to remove the stack trace.</li>
<li>It's the last stop in your application's stack and you want to log the exception before letting it propagate up or out. For example Application_Error in Global.asax.cs.</li>
<li>You want to handle all exceptions in a generic way instead of letting them reach the runtime engine.</li>
</ol>
<h4>
</h4>
<h4>
"Safety Net" Pattern for Background Threads</h4>
<div>
<br /></div>
<script src="https://gist.github.com/claq2/1b088980d52e2a8081d8.js"></script>
<br />
<div>
I once saw this described as the safety net pattern, but I can't find the term in general use. You can see where it gets its name, though. Another name might be the Pokeman pattern, as in, you gotta catch 'em all.</div>
<div>
<br />
<h4>
</h4>
<h4>
Translating Exceptions</h4>
</div>
<div>
<br /></div>
<script src="https://gist.github.com/claq2/b7a6d2abec1e79dca8e3.js"></script>
<br />
<div>
There are times you want to put some context around an exception for the next layer up so that they know which operation failed and whether there's something to be done about it. Include the current exception in the new one so that it gets put into the new exception's InnerException property. That way when it finally gets logged by something near the top of the call stack all of the error information will be present.</div>
<div>
<br />
A similar approach is to purposely not include the current exception in the new exception. You want to do this when you want to be sure that potentially sensitive details don't escape a layer. In this case, you want to consider logging the inner exception before it is lost to upper layer logging.<br />
<br />
<h4>
</h4>
<h4>
Centralized Logging and Handling</h4>
</div>
<div>
<br /></div>
<script src="https://gist.github.com/claq2/4ddf6f8e9be3aecbaa0a.js"></script>
<br />
<div>
<br /></div>
<div>
Some systems, like ASP.NET WebForms/MVC and WebAPI provide a way to catch all unhanded errors, at which point you can log and transform the output. You can also perform extra steps like emailing support. This can remove an awful lot of try/catch and error logging code from individual methods.<br />
<br /></div>
<h3>
</h3>
<h3>
Custom Exceptions</h3>
<div>
<br /></div>
<div>
Before creating a custom exception or a set of them you need to think about what you want to get out of them. Generally, a custom exception communicates that you've reached a very particular condition that you want to identify from other conditions. That way you can log it and trace it back to a single line of code and potentially let users know what they need to do.<br />
<br />
For example, business logic can be complex. It's usually a good idea to have a custom exception that represents a business layer failure, like not providing a needed bit of data. It's not enough, though, to throw this single exception type for every rule failure. You could hardcode the message to show e.g. "No IP address specified", "Invalid IP address specified", but then your app is limited to one language of user. (If you've ever used FxCop or Visual Studio's Code Analysis then you are probably familiar with the "move string to resource file" warning :)<br />
<br />
What's needed is one of the following:<br />
<br />
<ol>
<li>A collection of specific custom exceptions all inheriting from a layer-specific parent exception class. The parent class is for the convenience of the next layer up so that it can catch all layer-specific exceptions and handle them differently from other libraries' errors or .NET's.</li>
<li>A single layer-specific parent exception and a collection of error codes/enum values, each representing a particular condition.</li>
</ol>
<div>
An app can then map the collection of exceptions or the error codes/enum values to matching culture-specific strings. .NET provides Resource classes for this scenario.</div>
<div>
<br /></div>
<div>
If you use the error code/enum approach, put the value in the Data dictionary property that is part of the .NET Exception class. This is effectively a dictionary of <Object, Object> into which you can stuff whatever you like in order to send values up the call stack. Don't add properties to your custom exception. You may find yourself having to hand your custom exception to some generic handler that won't know about your properties. It will also save having to perform a slightly expensive cast to your custom type to get the property. Generic handlers will only need to query the Data property for the error code. Keep the key in a public static property.<br />
<br />
<script src="https://gist.github.com/claq2/89783206a1ee7814668d.js"></script>
</div>
<div>
<br /></div>
<div>
The alternative is an exception for each error instead of an enum value. Each exception should look up its own error message from a culture-specific Resource class. This approach is more in line with how .NET does things. Some drawbacks are the extra code and that the exceptions will only be understood by other .NET or SOAP-based systems. There is no standard way to send exceptions over, say, JSON.<br />
<br />
<script src="https://gist.github.com/claq2/3a7001d13880c56279df.js"></script>
This way you can send a language-specific message to the user and log in another language.<br />
<br /></div>
</div>
<h3>
</h3>
<h3>
Anti-patterns and Common Mistakes</h3>
<div>
<br /></div>
<div>
<h4>
</h4>
<h4>
Catch-Log-Throw</h4>
<div>
<br />
<script src="https://gist.github.com/claq2/7b795c4698ed90925965.js"></script>
</div>
Doing this all the way up the call stack will fill your logs with the same error again and again, each time adding another call to the stack. This makes logs much harder to read. Did the same error really happen just once? How do you accurately count the number of errors?<br />
<br />
Do centralize your logging and stick to the strategy. In the above case you would remove the try and the catch block. Just let the exceptions propagate up the stack.<br />
<br />
<h4>
</h4>
<h4>
Throw ex</h4>
<div>
<br /></div>
<script src="https://gist.github.com/claq2/ce127beb61da3322f049.js"></script>
To propagate a caught exception up the stack as-is, just "throw". When you "throw ex", you create a new exception with the same type as the caught one, but the call stack in the exception starts from where you threw it. The stack trace from the one that you caught is no longer available to upper layers. I don't know of a good reason to "throw ex". I've only ever seen it lead to poor debugging sessions and confusing logs.<br />
<br />
<h4>
</h4>
<h4>
Complex Exception Families</h4>
<div>
<br /></div>
Don't group your custom exceptions under other custom exceptions. For instance, if you have a SaveCustomerException and a LoadCustomerException, don't make them inherit from CustomerOperationException that inherits from a base CustomException class. It's needlessly complicated. No code needs to catch this vs. the specific exception or the base custom one.<br />
<br />
The worst approach I've seen is a combination of custom exceptions that each have their own embedded enum of reasons. Translating these into a single custom exception and single enum value ends up looking like this:<br />
<script src="https://gist.github.com/claq2/6c1672c03630de2c2cb0.js"></script><br />
Now every time there is a new exception reason in one of those enums this method needs another case statement. If there is a new class of exception to catch then we need a A total waste of time.<br />
<br />
<h4>
</h4>
<h4>
Not Using Exception.Data</h4>
<div>
<br /></div>
I've never seen .NET use the Data property, but it is the best place to put custom information, rather than adding properties to your custom exceptions. If you feel you must use custom properties, use Data as a backing store so that it is available without casting to your custom type.<br />
<br /></div>
</div>
James McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.com0tag:blogger.com,1999:blog-6792849853700107540.post-70054434005951099322014-11-21T23:00:00.000-05:002014-11-21T23:00:52.074-05:00Code is like Wet Sand Built on Wet SandThe softness of software means that everything we build is sitting on a <a href="http://en.wikipedia.org/wiki/Soil_liquefaction" target="_blank">liquefaction zone</a>. Everything works until someone decides to change the underlying structure. A NodeJS package changes and breaks your code. Entity Framework 6 removes a feature that you need and now your have to refactor your data layer. Chrome adopts the latest CSS revision and text now doesn't appear on your page.<br />
<br />
We need updates to software to fix bugs and make development tasks easier. If we didn't do that we'd still be developing with assembler and running DOS. We also need to retire old things because the combinations of things to support grow exponentially over time. Like that estimate before the invention of the transistor that we'd one day need every grown woman in America to be a phone operator, we'd need every high school grad to be a software tester to test all the combinations.<br />
<br />
The most common approach to keeping software flexible is to keep areas of functionality with an app very specific and to use abstractions so that no one part knows how another part works. In object oriented programming these are known as the <a href="http://en.wikipedia.org/wiki/SOLID_(object-oriented_design)" target="_blank">SOLID</a> principles.<br />
<br />
Now, if you and your team have the time, skill and discipline to obey these principles, your codebase can survive changes for many years without it getting much harder to modify. But companies need to make deadlines to make money, not everyone knows these ideas and sometimes people just want to be done with a task. Damn real world :)<br />
<br />
One way companies mitigate the amount of code they have to maintain is to drop support for things in their products. No more Windows XP and Internet Explorer 6 patches. jQuery 2.x and AngularJS 1.3 need IE9+. Otherwise the testing becomes too onerous, and makes paid products too expensive relative to their value. For free products, this is just too much to ask<br />
<br />
I think there is another approach taking shape: planned obsolescence at the app level, not just at the feature/platform level.<br />
<br />
The SOLID principles help keep it easy to replace tiny units of code within an app. Lately, though, it feels like people want to expand that approach to entire apps. I've heard people on coding podcasts ask JavaScript developers how they plan to keep a web app running for over 5 years when JS libraries change so fast and the answer is that they don't. They expect the app to be obsolete and to be replaced by something else.<br />
<br />
Every app that lives in a cloud system like Azure or Amazon Web Services costs someone some money for every moment it's running or every byte it delivers. Once an app's popularity wanes and the revenue stream drops below the cost of keeping it running it will be decommissioned. Online games are already doing this.<br />
<br />
You can see this happening with mobile platforms, too. It's easy to do with hardware that can't be upgraded and costs as much to repair as to replace with the latest version. I have a Windows Phone 7 phone and the flow of new apps for it has almost stopped. Your iPhone 4S can run iOS 8, but it probably shouldn't for performance reasons. As apps start requiring iOS 8+ your 4S and down will be left behind.<br />
<br />
As someone who works with some 10+ year old code at work, I support the idea of planned obsolescence at the app level. What would that look like for executables on PCs and servers?<br />
<br />
<ul>
<li>A set of small apps with very limited responsibilities. No monolithic services with 20 threads doing 20 different things.</li>
<li>Lots of executables. Maybe one that launches the rest, but again, no 1 giant process with a ton of threads.</li>
<li>Self-updating. Don't rely on users to keep things up to date. Provide control for the risk-averse enterprise, but limit component use to a year or 2.</li>
</ul>
<div>
Shortly after I started this post Martin Fowler's wrote about "<a href="http://martinfowler.com/bliki/SacrificialArchitecture.html" target="_blank">sacrificial architecture</a>", which seems to be a way to deal with retiring large scale systems. Great minds think alike? :)</div>
<br />
<br />James McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.com0tag:blogger.com,1999:blog-6792849853700107540.post-40862260906217882942014-10-11T14:35:00.000-04:002014-10-11T14:35:18.206-04:00How to Fund Childcare in CanadaMy idea is this: don't limit the childcare deduction to just people's own kids<br />
<br />
In Canada, childcare expenses are tax deductible in the same way that retirement fund (RRSP) contributions are. They come off the total of your income, so you get back whatever percentage your highest tax rate is. The more you make the more the deduction is worth. (Whoever has the lower income in a couple has to claim it.)<br />
<br />
I understand the reason an RRSP contribution comes off the top of your income. You're loosing access to money now in hopes that you will be in a lower tax bracket later in life. But I don't quite understand why childcare is treated the same way. Tax law history is not easy to find online. It's probably something to do with the idea that you spend $50 a day to go to work and make $150. You don't get to spend the $50 on yourself or on goods or experiences for your kids. (A good daycare can be a good developmental experience, so this reason doesn't make complete sense.)<br />
<br />
Whatever the reason I suggest we let people pay to send other people's kids to day care and let them deduct the cost in the same way. It would be a like a charitable contribution, but worth more than 15% to those in higher tax brackets.<br />
<br />
There are a couple of different ways you could run this. One would be to have to pay for a particular child. A group of family and friends would all pay a child's parent(s) some amount towards day care. There would have to be receipts from the child's parent.<br />
<br />
Another way would be to have non-profit companies to which anyone could contribute and parents would apply for help. You'd loose some of the contributions to the overhead of running the company, but the benefits would be more widespread.<br />
<br />
My next step is to write to all of the federal leaders and maybe even the provincial ones. I'm not a fan of the federal Conservatives, but it wouldn't be the first time that they extend a tax deduction. This might be right up their alley.James McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.com0tag:blogger.com,1999:blog-6792849853700107540.post-65138394093862162232014-05-26T21:08:00.000-04:002016-04-01T22:18:30.370-04:00Don't Be Afraid of SingleOrDefaultThe following is based on an issue I've seen many times in a code base at work, and only recently caused the problem described below.<br />
<br />
Consider the following code:
<br />
<script src="https://gist.github.com/claq2/38ad4e9f3a19801c2206.js"></script><br />
<br />
Assume that for good reasons we need to find JobSources by their Source and JobId values. The calling code doesn't have the Id. The "database" doesn't enforce uniqueness of these 2 values as a key, we just assume that every other system that feeds up data behaves this way, too. Well, hope is a better word, as we'll see.<br />
<br />
The first time UpdateOrAdd is called, it will find the one and only matching JobSource and update it. Then someone adds a second JobSource for job 2. We again try to update the JobSource with JobId 2 and SourceType File. Success! No exceptions.<br />
<br />
Except, we're using Any to see if there are 1 or more items and then updating the first one. The code tolerates the condition of there being more than one item that matches our "key". We have potentially updated the wrong data. Corruption!<br />
<br />
Now let's change UpdateOrAdd to use SingleOrDefault:<br />
<br />
<script src="https://gist.github.com/claq2/6c71666f3139efb4cfe6.js"></script><br />
<br />
Not only is the expected shape of the data clear, but it's shorter too. SingleOrDefault returns the 1 item that matches or the default value for the type. For any class type, the default is null. If more than 1 item matches, it throws an exception.<br />
<br />
Now on the second call to UpdateOrAdd, we get the following exception:<br />
<br />
System.InvalidOperationException: Sequence contains more than one element.<br />
<br />
Oh no! A noisy exception, instead of silent data corruption!<br />
<br />
Why would someone write code the earlier Any/First stuff? Because some coders are scared of triggering exceptions. More scared of that than corrupting data. Any time I see this pattern in code at work I change it to use SingleOrDefault, or just Single if the item is definitely expected to be present. I've had to convince people that it's better to report a problem than update the wrong item.<br />
<br />
Yes, there is a chance that the exception won't happen during testing because the data in the lab might be very clean. Perhaps no one has yet created the conditions there that break your assumption about the shape of the data. It just might be found by a real customer. That customer, though, will report the issue and, more importantly, <i>not lose data</i>.James McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.com0tag:blogger.com,1999:blog-6792849853700107540.post-2511201676258802892014-05-12T23:00:00.000-04:002014-05-12T23:00:53.080-04:00A Very Practical Use of C# StructsA programming interview question folks will tell you to be prepared for is "when should you use structs" and the general response is "when you want pass by value behaviour". That is, when you want data to be passed as a copy rather than as a reference.<br />
<br />
Ok, but when would you ever want that behaviour? An issue at work recently provided an example: when you can't trust that the thing your object was passed won't be changed. Or that the object you passed in turns out to be the same one passed out later. Here's an example:<br />
<br />
<br />
<script src="https://gist.github.com/claq2/05839fadb9a862680342.js"></script>
<br />
There is a class, PersonUsingClasses, that takes items from two data sources and holds on to the references, rather than making copies of all of the properties of each. It then uses those references to construct a new property, DisplayName. I've written such code myself, unaware of the dangers. This class also has a way to convert it back to one of the data source types, ToServicePerson.
<br />
<br />
A couple of things to notice:
<ol><li>The PersonClassFromService reference sent into the constructor is the same one passed out in ToServicePerson</li>
<li>Someone forgot to remove his debugging code in the constructor.</l1></ol>
This means that any changes made to the Service person object inside or outside of the PersonUsingClasses object will appear everywhere at once. Frank changed his salutation in Main to Dr. from outside the class, but the class reflected this change. Also, some code in Main tries to detect if the Age property changed by comparing the input and output objects, and what do you know, there was no change.
<br />
<br />
One option to fix this is to not hold on to the references passed into the constructor and make copies of every property that matters to PersonClassUsingClasses. That might be a good option, but we need to support recreating the PersonClassFromService object, which could have properties that don't matter. Or, there could be dozens and dozens of properties to track. (I worked at a bank - trust me when I say this happens.)
<br />
<br />
A nearly equivalent approach is to use return structs from the DB and service:
<br />
<br />
<script src="https://gist.github.com/claq2/dbd143131493e356887d.js"></script>
<br />
Now the system copies every property in every input parameter for us because they are structs. It copies every property again when we call ToServicePerson. This means that changing the input parameter does not get reflected in PersonUsingStructs and we can detect the accidental change in the constructor. This approach uses more memory because of all of this copying, but it's not much more than if we'd made copies of every property ourselves in PersonUsingClasses. We've protected the system from accidental changes.<br />James McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.com0tag:blogger.com,1999:blog-6792849853700107540.post-48210539306548935252013-05-15T22:48:00.000-04:002013-05-15T22:48:53.455-04:00Azure Mobile Services HTML Client - This Changes EverythingI started building a web app for tracking my budget. It stores data in a SQL database, so I thought I'd try running it in Microsoft's Azure cloud system. Azure has a web hosting system that is very easy to use and a SQL database system (among many, many other things). The architecture looked like this:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://docs.google.com/drawings/d/1YVeiKFxAqe1Uu1yt6yUXYznle8ywgRPibpKEEZ3RYUY/pub?w=483&h=132" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img alt="Website and database" border="0" src="https://docs.google.com/drawings/d/1YVeiKFxAqe1Uu1yt6yUXYznle8ywgRPibpKEEZ3RYUY/pub?w=483&h=132" title="Website and database" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Website and database</td></tr>
</tbody></table>
<br />
My first build of the site didn't use authorization and the data in the database couldn't be associated with a particular user. Before .NET 4.0 the out-of-the-box web authentication system was web forms. This meant storing user IDs and passwords in a database and forcing users to create yet another set of credentials on your site.<br />
<br />
However, the ASP.NET MVC 4 template leveraged the use of claims-based authentication in .NET 4 and gave you an easy way to let users log in with third party IDs. The included ones are Google, Facebook, Twitter and Microsoft IDs (aka Live IDs). This sounded good to me, so I started down the path of moving my app to the new MVC 4 template. I could log in with my Google ID and received a unique identifier with which to associate my data. The architecture was now:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://docs.google.com/drawings/d/10bt6BrHdQaNPGmB7OInXkyBrkgmFc8RYVtrdXFU-bGA/pub?w=276&h=220" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img alt="Website, database and authorization" border="0" src="https://docs.google.com/drawings/d/10bt6BrHdQaNPGmB7OInXkyBrkgmFc8RYVtrdXFU-bGA/pub?w=276&h=220" title="Website, database and authorization" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Website, database and authorization systems</td></tr>
</tbody></table>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
Shortly after that, I learned about Azure Mobile Services. This is a very handy system that can handle authentication with the same systems as the MVC 4 template, but for smart phones and Windows 8 Store apps. It also integrates with an Azure SQL database. My intention was to use it to create Windows 8 and Windows Phone 7 and 8 clients for my app.<br />
<br />
I found out that, while it is technically possible to use an existing database with Azure Mobile Services, there are <a href="http://chrisrisner.com/Connecting-an-Existing-Database-to-Windows-Azure-Mobile-Services">some hoops through which you must jump</a> to make them talk. I wasn't too far along with my development so I thought it easier to allow the Mobile Service creation wizard to create the database. Soon, I had a little dummy Windows 8 app that could authenticate with my Google ID, and again was receiving a unique ID.<br />
<br />
Only, it wasn't the same as the ID I received in the web app. This was a problem, because it meant I couldn't look up data for the same person using their Google ID in both apps. My architecture now looked like this:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://docs.google.com/drawings/d/1SEBA0CMSmZAIMtsEF2La4s_0wvGHCVfYp9Q6alxqQJ8/pub?w=486&h=310" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img alt="Website, database, mobile services and 2 authorization systems" border="0" src="https://docs.google.com/drawings/d/1SEBA0CMSmZAIMtsEF2La4s_0wvGHCVfYp9Q6alxqQJ8/pub?w=486&h=310" title="Website, database, mobile services and 2 authorization systems" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Website, database, mobile services and 2 authorization systems</td></tr>
</tbody></table>
The cause of the ID mismatch was that the MVC4 template uses Google's OpenID authentication system, while Mobile Services uses Google's OAuth2 system. I had no reliable way of associating one with the other. A little searching around found other people encountering the same problem, and eventually a <a href="https://nuget.org/packages/DotNetOpenAuth.GoogleOAuth2">NuGet package</a> that lets you use Google's OAuth2 system in an MVC 4 web app.<br />
<br />
The NuGet package would let me find people's data no matter which app they used. Hardly ideal, though, because it meant extra code just to do the same thing. I suspected there would be other differences between the MVC 4 template's authentication providers and Mobile Services' providers.<br />
<br />
My ideal architecture is:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://docs.google.com/drawings/d/1o4xXh3PkJjk0ZDrGkrL03bxVIRnehFDM78nKi9OKPus/pub?w=361&h=310" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://docs.google.com/drawings/d/1o4xXh3PkJjk0ZDrGkrL03bxVIRnehFDM78nKi9OKPus/pub?w=361&h=310" /></a></div>
<br />
<br />
The idea is that the Mobile Service would handle all authorization, and then access a REST endpoint in the website, rather than going directly to the database. This means that the web app would somehow authenticate against and get data from the Mobile Service, just like the other apps.<br />
<br />
Besides the centralization of the authorization mechanism, it would also put all the business logic on the server. The Windows 8 and phone apps would be strictly for displaying data. That way any bug fixes to business logic would be web-based, meaning immediate and global. If the logic lived in the Windows and phone apps then I'd have to update them individually, and wait for them to be approved and delivered to people.<br />
<br />
It seemed like a pipe dream, because I couldn't find any easily understandable information about how to authenticate against a Mobile Service. Until a few days ago, when I found out that the Mobile Services group published an <a href="http://www.windowsazure.com/en-us/develop/mobile/tutorials/get-started-html/">example HTML client</a>, along with a <a href="http://msdn.microsoft.com/en-us/library/windowsazure/jj554207.aspx">JavaScript library</a> that can authenticate and get data. <br />
<br />
This changes everything.<br />
<br />
This is the final piece of the puzzle that will let me create a web app that gets data through Azure Mobile Services.<br />
<br />
Actually, there is one change to the architecture needed to keep the REST interface accessible to only the Mobile Service:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://docs.google.com/drawings/d/1sCDlnqKgsUOX0vRu7GUUG-M6VkYzmfg2A-CO32HCsL8/pub?w=486&h=310" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://docs.google.com/drawings/d/1sCDlnqKgsUOX0vRu7GUUG-M6VkYzmfg2A-CO32HCsL8/pub?w=486&h=310" /></a></div>
<br />
<br />
That extra User Auth box is an Azure Access Control Service service ID that the Mobile Service will use to access the web service's REST service.<br />
<br />
<br />James McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.com1tag:blogger.com,1999:blog-6792849853700107540.post-87605781643197458502013-01-19T22:39:00.000-05:002013-01-19T22:39:26.697-05:00The First Bank of JamesAfter listening to economics podcasts for the last couple of years, and recent poor customer experiences at several banks, I've decided to open my own Canadian bank.<br />
<br />
Well, not really because I'm sure the cost due to red tape would dissuade all but the deepest pockets, and a bunch of my ideas are currently forbidden. I was mostly inspired by <a href="http://www.econtalk.org/archives/2008/11/selgin_on_free.html">one about free banking in Scotland between 1716 and 1845</a>. Free, as in, no regulation and no government to bail you out. So, my fantasy bank would:<br />
<ul>
<li>Not trade its stock publicly, unless additional conditions could be placed upon the board beyond maximizing shareholder value (I don't know if this is possible under Canadian law). Such conditions would include not buying or selling any securities outside stocks and bonds, and being able to say no to investments that don't fit our social conscience. And no selling of the bank to another bank in part or whole.</li>
<li>Require all board members and insiders to hold at least 75% of each of their deposit (e.g chequing, savings), credit and investment values in the bank.</li>
<li>Opt out of the Canadian Deposit Insurance Commission (is this possible?). If the bank fails, people may lose money, including board members. We will be motivated to not lose our and your money because there will be no backup plan. (CIDC guarantees that if a bank fails every deposit account will be paid out up to $100000. Want us risking the whole bank knowing customers will be bailed out?)</li>
<li>Offer only index mutual funds. Canadian index, US index, international index, a couple of bond index funds, maybe a gold one. No one can beat the market consistently, so no one in my bank will try.</li>
<li>Sell only our mutual funds through fee-only financial planners. That is, you'll pay our people a one time, hourly fee for financial advice. The reason you don't pay the planner at your current bank anything to set you up in mutual funds is because he or she gets paid by the funds your buy themselves. Also known as commission sales. This means that the cost of running the mutual finds will be greatly reduced. Those commission sales are paid for by the x% taken out of funds every year. Canada has the highest average fees of all industrial countries.</li>
<li>Not be a member of the Mutual Fund Dealers Association (MFDA) (not possible under current laws). We'd go to real lengths to build you a portfolio appropriate to your needs and risk tolerance without being able to hide behind the protections of this group. No doing the MFDA minimum needed to survive a court case, because we'd be vulnerable to all legal action. This would also make buying mutual funds as easy as any internet purchase because we wouldn't be beholden to the MFDA's documentation rules. You'd need to jump through our hoops, which would align with your actual needs, but these would not involve filling out pages of generic forms. You</li>
<li>Not be regulated by the provincial securities exchanges. Completely impossible, I know, but it would also make buying stocks, bonds and mutual funds as easy as a couch. Far less paperwork. The bank would be forced to perform its own due diligence on transactions, rather than the minimum requirements currently needed.</li>
<li>Keep the external technological systems up to date in terms of technology and features. Phone apps, third party integration, a slick web interface, analysis tools inspired by the latest financial sites will all be standard. A modern look and feel replicated across the whole site, not just the "marketing", or non-personalized, site. We know you you are - no need to fill in blank paper forms. No PDFs allowed. No emails saying "if you are registered for web banking... if you have an investment account..." because we know whether you are/do or not. The goal is to gather any given bit of info once and only once.</li>
<li>Maintain one, and only one, copy of your personal information, guarded by the leading edge in encryption, processes, ninjas, balrogs, whatever it takes. No physical copies allowed that could wander into the wilderness, like tapes. Backups would be handled over the internet, encrypted at the source. Business continuity would be a priority because of point 2 above.</li>
<li>Be very conservative in its lending. Mortgages would require the traditional minimum 20% down. No Canadian Mortgage and Housing Commission (CMHC) insurance allowed for high ratio mortgages. Other lending would require very solid documentation and low risk levels. The board is risking most of its money on your loan, so will allow only the most solid investments.</li>
<li>Not have call centers. You'd talk to people at the local branch on the phone who know you and your significant other and kids. This would be partly for the personal touch and a security measure.</li>
<li>Structure all technological and business systems around the customer. No isolated stovepipe departments to make internal interactions harder.</li>
</ul>
One can dream.<br />
<ul>
</ul>
James McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.com0tag:blogger.com,1999:blog-6792849853700107540.post-55384171467699463602013-01-11T23:09:00.002-05:002013-01-11T23:44:55.862-05:00Want First World Problems? Buy a New SSDBuy a new solid state drive (SSD). In my case, a 128GB OCZ Vertex 4, which is well reviewed.<br />
<br />
And then learn that it supports SATA 3 (the interface between the drive and the rest of the computer).<br />
<br />
And then learn that your motherboard only supports SATA 2.<br />
<br />
And then learn from Wikipedia that this should be backwards compatible, but with the SATA 2 speed. Accept this limitation.<br />
<br />
And then, after an evening of experimenting, that your Gigabyte MA785GMT-UD2H motherboard won't see the new drive unless it's in IDE mode, which is the oldest, slowest mode available.<br />
<br />
And then spend another evening googling, finding that:<br />
<ul>
<li>AMD (chip manufacturer) decided to rely on motherboard makers to do things right with the 785G chipset, rather than taking defensive steps themselves</li>
<li>other people have had the same problem with similar Gigabyte 785G boards, and had to request BIOS updates through their support site</li>
</ul>
And then submit a support ticket for your 2 year old Gigabyte board about this issue.<br />
<br />
And then visit a Tiger Direct store to get a SATA 3 controller add-on card, but find they are all out.<br />
<br />
And then visit a Best Buy for the same thing, but find they have no clue what the hell you're talking about. Give them the part number from their website. Find out they don't have it in their stores.<br />
<br />
And then wait a couple of business days with no action from Gigabyte.<br />
<br />
And then decide that this is the first, last and only Gigabyte/AMD motherboard that you will ever buy. The next full system overhaul will involve an ASUS board and Intel everything else. In the meantime, order an ASUS M5A97 LE R2.0 motherboard that supports your current memory, CPU and SATA 3. This order is non-refundable, but is exchangeable for store credit from the very excellent TigerDirect.ca.<br />
<br />
And then the next day find Gigabyte has responded to your support ticket with a new BIOS that makes the board see the new drive in SATA 2 mode.<br />
<br />
And then wait a week for your new motherboard. Notice that Gigabyte hasn't posted the new BIOS on their website, implying the update is available only to those opening support tickets.<br />
<br />
And then get most of the way swapping boards before realizing the new board doesn't have an old-style IDE connector for your DVD drive. Luckily you have your parents-in-law's old computer that has a DVD drive with a SATA connector that you can use. (The board also has no floppy drive connector, which is fine.)<br />
<br />
Also order 2 different kits for mounting 2.5" SSDs in a 3.5" drive bay, both of which aren't drilled for the "old" style 3.5" drive screws. Realize that the mounting kit currently in the PC, which is drilled properly, can hold a second SSD.<br />
<br />
Finally get everything connected and working. Feel no great satisfaction.<br />
<br />
First world problems, to be sure. No one died, no one got sick or injured, no massive loss of property. But, does it have to be this hard?<br />
<br />James McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.com0tag:blogger.com,1999:blog-6792849853700107540.post-20050425937483593352012-11-24T17:21:00.001-05:002012-11-24T17:21:36.066-05:00TD e-Series Post Follow Up<a href="http://jamesmclachlan.blogspot.ca/2012/11/a-letter-to-td-canada-trust-on-their.html">This post</a> got me some personal attention from TD after I posted a link on Twitter. Their social game is pretty good. They ask me for some contact info and after a few days someone in the e-Series group contacted me. She was very nice and we had a chat about the process as it stands and how it could be better.<br /><br />
It turns out I used the wrong form altogether. The 11 page one that I've spent so much time on is for opening a new account. There is a much shorter one for converting an existing account. Big, big sigh. I'm really not sure how I missed that. Now that I go looking for the forms I see that there is always a link for both cases. Maybe having not worked at CIBC since February my form-fu has gotten rusty.<br />
<br />
According to the person I spoke to, there are a couple of interesting reasons TD doesn't offer the e-Series index mutual funds on all mutual funds, making the extra steps necessary:<br />
<ul>
<li>Due to mutual fund regulations enforced by the Mutual Fund Dealers Association these funds shouldn't be offered by fund advisers due to their "do it yourself" nature. I'm not totally convinced this is a major factor because TD, like the other banks, offer a range of index funds through advisers, but with higher management expense ratios (MERs).</li>
<li>Fund advisers get commissions from selling mutual funds, which is why even normal index funds have surprisingly high MERs. They can't sell the e-Series because there is no compensation structure, hence the lower MERs. To me, this is the primary reason.</li>
</ul>
I made a few suggestions about the process, like somehow pre-filling the forms with information after you've logged in to online banking. Another was to just make the e-Series funds available to everyone all the time. I understand that they can't be sold through a branch, but I'm not sure what would prevent making them available to everyone online only, I didn't ask. Perhaps some compensation happens even when you buy mutual funds online that needs to change.<br />
<br />
I'm happy that someone from TD listened to my suggestions, and seemed to understand that the process could be better. She offered to help speed up the process by having me fax the forms to her, which is very nice. But I still have a bit of a bad taste left in my mouth having had to try this 3 times for the same account. It's made me very curious about potential alternatives.<br />
<br />
All Canadian banks suffer from process problems. So far President's Choice Financial has been the lowest friction one I've dealt with, because of their online nature. If ever foreign banks are allowed to compete seriously in Canada the current banks will have to adapt very quickly. Things still feel like they're stuck in a time before the internet.<br />
<br />James McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.com1tag:blogger.com,1999:blog-6792849853700107540.post-49262388264099602842012-11-23T00:17:00.000-05:002012-11-23T00:17:07.233-05:00My Investment Plan - Index and IgnoreThe last few times I've set up investment accounts at TD I've had to sit through their "Investor Profile" quiz. I imagine most banks do this, too. They ask you a series of questions to see if how you think of yourself in terms of investing matches how you'd react to certain situations. For example, if you say you want a highly risky, all-stocks portfolio, but couldn't stomach a 5% loss in a year, then you're inconsistent. If you are consistent then they will recommend a portfolio that better matches your true risk tolerance.<br />
<br />
However, if your plan is to stick to a certain asset allocation year after year, not changing it for any reason, they have no clear way to accommodate you. As I recall it, there is yet another form that explicitly absolves them of any differences between your answers and what you do with your money. It's all about identifying potential liabilities in the form of people whose portfolios don't match their risk tolerance. The assumption is that everyone needs help investing from the mutual fund salesperson. Oh, did you not realize everyone in the branch is a salesperson, including the tellers? Opening an account gets someone some money, and getting you into specific mutual funds is very valuable.<br />
<br />
<br />
Doctors don't scan your thyroid for potential cancer activity because there is so much weird stuff in thyroids that there's no point in acting on the findings. And so it goes for index investing. My plan is to follow a strategy of investing in very low cost index
funds, rebalancing to a particular asset allocation yearly with a net deposit. This is
called couch potato portfolio investing because it involves so little effort. I pay no attention to what's going on with my money because
there is no information on which I would act. Up
15%? Whatever. Down 30%? Don't care. (Actually, if I knew this I'd pile in more money because it's like stuff's on sale.) Index investing addresses the two biggest impacts to investing: 1. the fees in the form of the management expense ration (MER) and 2. the damage done by investors selling when things are low and buying when things are high.<br />
<br />
As a math geek I can envision myself tracking the daily value and yearly return of my (still small) portfolio, just because I know how. However, the information is as meaningless to me as the standings in the NFL. I'm happy to let that which doesn't matter truly slide, as Tyler Durden does.<br />
<br />James McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.com0tag:blogger.com,1999:blog-6792849853700107540.post-29022553821056701022012-11-18T22:45:00.002-05:002012-11-18T22:45:26.287-05:00A Letter to TD Canada Trust on their Abysmal e-Series Application Process<!--[if gte mso 9]><xml>
<o:OfficeDocumentSettings>
<o:AllowPNG/>
</o:OfficeDocumentSettings>
</xml><![endif]--><br />
<!--[if gte mso 9]><xml>
<w:WordDocument>
<w:View>Normal</w:View>
<w:Zoom>0</w:Zoom>
<w:TrackMoves/>
<w:TrackFormatting/>
<w:PunctuationKerning/>
<w:ValidateAgainstSchemas/>
<w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid>
<w:IgnoreMixedContent>false</w:IgnoreMixedContent>
<w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText>
<w:DoNotPromoteQF/>
<w:LidThemeOther>EN-US</w:LidThemeOther>
<w:LidThemeAsian>X-NONE</w:LidThemeAsian>
<w:LidThemeComplexScript>X-NONE</w:LidThemeComplexScript>
<w:Compatibility>
<w:BreakWrappedTables/>
<w:SnapToGridInCell/>
<w:WrapTextWithPunct/>
<w:UseAsianBreakRules/>
<w:DontGrowAutofit/>
<w:SplitPgBreakAndParaMark/>
<w:EnableOpenTypeKerning/>
<w:DontFlipMirrorIndents/>
<w:OverrideTableStyleHps/>
</w:Compatibility>
<m:mathPr>
<m:mathFont m:val="Cambria Math"/>
<m:brkBin m:val="before"/>
<m:brkBinSub m:val="--"/>
<m:smallFrac m:val="off"/>
<m:dispDef/>
<m:lMargin m:val="0"/>
<m:rMargin m:val="0"/>
<m:defJc m:val="centerGroup"/>
<m:wrapIndent m:val="1440"/>
<m:intLim m:val="subSup"/>
<m:naryLim m:val="undOvr"/>
</m:mathPr></w:WordDocument>
</xml><![endif]--><!--[if gte mso 9]><xml>
<w:LatentStyles DefLockedState="false" DefUnhideWhenUsed="true"
DefSemiHidden="true" DefQFormat="false" DefPriority="99"
LatentStyleCount="267">
<w:LsdException Locked="false" Priority="0" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Normal"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="heading 1"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 2"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 3"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 4"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 5"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 6"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 7"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 8"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 9"/>
<w:LsdException Locked="false" Priority="39" Name="toc 1"/>
<w:LsdException Locked="false" Priority="39" Name="toc 2"/>
<w:LsdException Locked="false" Priority="39" Name="toc 3"/>
<w:LsdException Locked="false" Priority="39" Name="toc 4"/>
<w:LsdException Locked="false" Priority="39" Name="toc 5"/>
<w:LsdException Locked="false" Priority="39" Name="toc 6"/>
<w:LsdException Locked="false" Priority="39" Name="toc 7"/>
<w:LsdException Locked="false" Priority="39" Name="toc 8"/>
<w:LsdException Locked="false" Priority="39" Name="toc 9"/>
<w:LsdException Locked="false" Priority="35" QFormat="true" Name="caption"/>
<w:LsdException Locked="false" Priority="10" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Title"/>
<w:LsdException Locked="false" Priority="1" Name="Default Paragraph Font"/>
<w:LsdException Locked="false" Priority="11" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtitle"/>
<w:LsdException Locked="false" Priority="22" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Strong"/>
<w:LsdException Locked="false" Priority="20" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Emphasis"/>
<w:LsdException Locked="false" Priority="59" SemiHidden="false"
UnhideWhenUsed="false" Name="Table Grid"/>
<w:LsdException Locked="false" UnhideWhenUsed="false" Name="Placeholder Text"/>
<w:LsdException Locked="false" Priority="1" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="No Spacing"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 1"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 1"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 1"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 1"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 1"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 1"/>
<w:LsdException Locked="false" UnhideWhenUsed="false" Name="Revision"/>
<w:LsdException Locked="false" Priority="34" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="List Paragraph"/>
<w:LsdException Locked="false" Priority="29" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Quote"/>
<w:LsdException Locked="false" Priority="30" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Quote"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 1"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 1"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 1"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 1"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 1"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 1"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 1"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 1"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 2"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 2"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 2"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 2"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 2"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 2"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 2"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 2"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 2"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 2"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 2"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 2"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 2"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 2"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 3"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 3"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 3"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 3"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 3"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 3"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 3"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 3"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 3"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 3"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 3"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 3"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 3"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 3"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 4"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 4"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 4"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 4"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 4"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 4"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 4"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 4"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 4"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 4"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 4"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 4"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 4"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 4"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 5"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 5"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 5"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 5"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 5"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 5"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 5"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 5"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 5"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 5"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 5"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 5"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 5"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 5"/>
<w:LsdException Locked="false" Priority="60" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Shading Accent 6"/>
<w:LsdException Locked="false" Priority="61" SemiHidden="false"
UnhideWhenUsed="false" Name="Light List Accent 6"/>
<w:LsdException Locked="false" Priority="62" SemiHidden="false"
UnhideWhenUsed="false" Name="Light Grid Accent 6"/>
<w:LsdException Locked="false" Priority="63" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 6"/>
<w:LsdException Locked="false" Priority="64" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 6"/>
<w:LsdException Locked="false" Priority="65" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 1 Accent 6"/>
<w:LsdException Locked="false" Priority="66" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium List 2 Accent 6"/>
<w:LsdException Locked="false" Priority="67" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 6"/>
<w:LsdException Locked="false" Priority="68" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 6"/>
<w:LsdException Locked="false" Priority="69" SemiHidden="false"
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 6"/>
<w:LsdException Locked="false" Priority="70" SemiHidden="false"
UnhideWhenUsed="false" Name="Dark List Accent 6"/>
<w:LsdException Locked="false" Priority="71" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Shading Accent 6"/>
<w:LsdException Locked="false" Priority="72" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful List Accent 6"/>
<w:LsdException Locked="false" Priority="73" SemiHidden="false"
UnhideWhenUsed="false" Name="Colorful Grid Accent 6"/>
<w:LsdException Locked="false" Priority="19" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtle Emphasis"/>
<w:LsdException Locked="false" Priority="21" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Emphasis"/>
<w:LsdException Locked="false" Priority="31" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Subtle Reference"/>
<w:LsdException Locked="false" Priority="32" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Intense Reference"/>
<w:LsdException Locked="false" Priority="33" SemiHidden="false"
UnhideWhenUsed="false" QFormat="true" Name="Book Title"/>
<w:LsdException Locked="false" Priority="37" Name="Bibliography"/>
<w:LsdException Locked="false" Priority="39" QFormat="true" Name="TOC Heading"/>
</w:LatentStyles>
</xml><![endif]--><!--[if gte mso 10]>
<style>
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:"";
mso-padding-alt:0in 5.4pt 0in 5.4pt;
mso-para-margin-top:0in;
mso-para-margin-right:0in;
mso-para-margin-bottom:10.0pt;
mso-para-margin-left:0in;
line-height:115%;
mso-pagination:widow-orphan;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;}
</style>
<![endif]-->
<br />
<div class="MsoNormal">
I recently applied to be able to buy TD's e-Series mutual funds in my daughter's RESP account. These are index mutual funds with very low expense ratios, and are a great alternative to exchange traded index funds for small portfolios.</div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
I've been through the process 3 previous times with our other investing accounts (2 RRSPs, son's RESP). It's ridiculous, and getting more so every year. You go to a branch, fill in forms, answer questions and they open your new mutual fund account. Except you can't access the e-Series yet. You have to fill out something that looks just like a new mutual fund account application. It's really not obvious that this will result in the funds being available for purchase in your existing fund account.</div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
TD sent my application back because of some missing information, so I'm including this note with my response. At least they sent a prepaid envelope in which to return it.</div>
<div class="MsoNormal">
<br /></div>
<br />
<blockquote class="tr_bq">
<div class="MsoNormal">
To Whom It May Concern:</div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
You, TD Canada Trust, have made the process of being able to
buy your e-Series index mutual funds in an existing account about as hard as
possible. There is no reason to require so much information that you already
have. You already know my and my wife’s name, employment information, SINs,
banking information, and investment profiles. We gave it when we opened the
mutual fund accounts that we already own. Another bank is going to create a
very similar line of products, put a simple sign up process in front of it and
will eat your lunch, if they haven’t already. I just want the funds to appear
in the drop down list on the website. Why does it require 11 pages of
documentation? It verges on the level of a tax return. And I can file that
online!</div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
This is the fourth time we have been through this ridiculous
process, but it has never been this bad. I will not forget this experience. I will
have a hard time recommending the e-Series funds to others because of it.
MoneySense magazine often recommends them, but I will be writing a letter to
the editor to let them know just how onerous you have made the process. I plan
to research alternatives to these mutual funds for my future investing needs.</div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Sincerely,</div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
James McLachlan</div>
</blockquote>
James McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.com0tag:blogger.com,1999:blog-6792849853700107540.post-77875986682395433282012-04-28T11:26:00.001-04:002012-04-28T14:39:52.462-04:00Running Tests, Boodhoo StyleWatch <a href="http://www.dnrtv.com/default.aspx?showNum=213">this</a> and weep at the beauty of <a href="http://blog.developwithpassion.com/">Jean-Paul Boodhoo</a>’s development environment. In particular marvel at the whole “saving a file causes compilation, runs tests and lists failures in a toast pop-up” thing. WANT.<br />
JP publishes everything you see in the dnrTV episodes on <a href="https://github.com/developwithpassion/dnrtv_dotnet_bootcamp_prep">github</a>, so we can all learn. He <a href="http://blog.developwithpassion.com/2012/03/28/webinar-devtools-an-walkthrough-of-my-devtools-setup/">recorded a webinar</a> about his setup, but hasn’t published the video yet. I couldn’t wait so I went digging.<br />
At the core is a Ruby script that checks every second for updated files and runs msbuild to compile and then the MSpec test runner. The script parses the MSpec runner’s output for failures and sends the text to Growl for Windows, which produced the toast notification. It even customizes the icon that Growl to use depending on whether there are failures or not – a fiery (red) Mario for failures and a happy Mario for success.<br />
It’s a somewhat complex system that does a few other things, like dissolving each subfolder in the main project into separate assemblies and batches some git commands. I was particularly interested in implementing the Growl notification with the failed tests.<br />
My setup:<br />
<ol>
<li><a href="http://www.growlforwindows.com/gfw/default.aspx">Growl for Windows</a> installed in c:\utils\growl for windows </li>
<li>JP’s Mario pictures (<a href="https://github.com/developwithpassion/dnrtv_dotnet_bootcamp_prep/blob/master/build/tools/growl/red.jpg">red.jpg</a>, <a href="https://github.com/developwithpassion/dnrtv_dotnet_bootcamp_prep/blob/master/build/tools/growl/green.jpg">green.jpg</a>) from github copied to c:\utils\growl for windows. </li>
<li><a href="http://teamcity.codebetter.com/guestAuth/repository/download/bt342/.lastSuccessful/Machine.Specifications-Release.zip">MSpec</a> extracted to C:\utils\Machine.Specifications-Release. </li>
<li>runspecs.bat (content below) added to the tests project in my solution and set to always be copied to the output folder. </li>
<li>A post build command on the tests project:<br />$(TargetDir)runspecs.bat $(TargetFileName)</li>
</ol>
Now, every time you build your solution tests will run and failures, or a success message, are sent to Growl.<br />
The batch:<br />
<code>@echo on<br />setlocal EnableDelayedExpansion<br />C:\utils\Machine.Specifications-Release\mspec-clr4.exe --html .\specs.html %1|find "FAIL">fail.txt<br />set fail=<br />for /f "tokens=*" %%a in (fail.txt) do set fail=!fail!\n%%a<br />if not "!fail!"=="" "c:\utils\growl for windows\growlnotify.exe" /t:Build /i:.\red.jpg "!fail!"<br />if "!fail!"=="" "c:\utils\growl for windows\growlnotify.exe" /t:Build /i:.\green.jpg "All passed"</code> <br />
Visual studio passes the name of the DLL with the tests in the variable %1. The batch asks mspec-cl4.exe to create an html report of the test results into specs.html (for later examination if needed), and also picks out the lines in mspec-cl4.exe’s console output containing “FAIL”. Those lines are piped to fail.txt.<br />
Next, it for loops through the lines in fail.txt and builds a batch variable called fail. Each line in fail.txt is separated by a “\n” so that Growl puts each one on a separate line. The !fail! syntax, alone with the setlocal EnableDelayedExpansion command, causes the batch processor to update the contents of the fail variable each time through the loop. The usual %fail% syntax won’t do this.<br />
Finally the batch decides if there are any failures or not and sends the appropriate message to growlnotify.exe.<br />
<a href="http://lh6.ggpht.com/-1PARsoWj5ww/T5wMBSZwq-I/AAAAAAAAAI0/YGW5_TTNIa4/s1600-h/image%25255B6%25255D.png"><img alt="image" border="0" height="134" src="http://lh4.ggpht.com/-Bn5RsN3CPNU/T5wMCCAjyII/AAAAAAAAAI8/elWUMzJaswQ/image_thumb%25255B2%25255D.png?imgmax=800" style="background-image: none; border-bottom: 0px; border-left: 0px; border-right: 0px; border-top: 0px; display: inline; padding-left: 0px; padding-right: 0px; padding-top: 0px;" title="image" width="262" /></a><br />
<a href="http://lh4.ggpht.com/-ZmdjAcspvF4/T5wMCXzW8vI/AAAAAAAAAJE/x24UcBz4rwo/s1600-h/image%25255B7%25255D.png"><img alt="image" border="0" height="134" src="http://lh6.ggpht.com/-3VBMlBlWxnQ/T5wMC0t20nI/AAAAAAAAAJM/ipZov4LclIs/image_thumb%25255B3%25255D.png?imgmax=800" style="background-image: none; border-bottom: 0px; border-left: 0px; border-right: 0px; border-top: 0px; display: inline; padding-left: 0px; padding-right: 0px; padding-top: 0px;" title="image" width="262" /></a><br />
Now to learn VIM and be a keyboard junkie like JP!James McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.com0tag:blogger.com,1999:blog-6792849853700107540.post-43687677774075097152012-04-27T07:20:00.001-04:002012-04-27T13:49:03.524-04:00I Can’t Say the Ford E-250 Is A Pleasure To Drive, But It Does Its Job Well<p>Ikea delivery will be $169? Bah! We can do better than that ourselves, right? Let’s rent a van and go get the stuff ourselves.</p> <p>Such was our plan one recent Sunday morning.</p> <p>You can rent a <a href="http://www.uhaul.com/Reservations/EquipmentDetail.aspx?model=BE">cargo van</a> at U-Haul for <font size="4"><strong>$19.95</strong></font> <font size="1">plus $0.49/km</font>. Great, just $20!</p> <p>Wait, what’s that small print? Oh, $0.49/kilometer. It’s about 30 km to the Ikea in Etobicoke, so 30 km x 2 (there and back) x $0.49 = $29.4.</p> <p>So we’re up to $50.</p> <p>Our plan was to take the whole family to Ikea, have lunch there, let our son play in the kids area and get some big pieces of furniture. If all went according to plan, we’d all drive to the U-Haul, I’d get the van, and we’d all drive on to Ikea.</p> <p>Only, we’re a little behind schedule. How to get me to U-Haul while mummy gets the kids ready? Take a cab.</p> <p>We’re now up to $75.</p> <p>One could argue that this wasn’t a direct, unavoidable cost, but it is something I had to pay for in order to get this furniture.</p> <p>At the U-Haul all went very well. This was a Sunday and it was not busy at all. The guy at the counter said that the previous day would have seen me waiting over an hour. Pro tip #1: don’t rent at U-Haul on Saturday.</p> <p>I decided to rent a dolly to move a big piece of furniture already in the house. That’s another $7. (Really? A $40,000 van for $20, a $100 dolly for $7? I think I’ll start a business renting just dollies.)</p> <p>The U-Haul guy offered me some optional insurance on the van for about $17. I had no idea if my car insurance or if paying by Visa covers me, so I bought that.</p> <p>So all totaled, U-Haul will cost $20 + $30 (mileage) + $7 (dolly) + $17 (insurance) + HST = $83.62. Add in the cab ride and our total so far is $108.62.</p> <p>And I have to replace the gas I used before I drop off the van. Yay.</p> <p>Off to Ikea in a bill board on wheels on a windy day. This isn’t a just a big minivan. This is a <a href="http://en.wikipedia.org/wiki/Ford_Super_Duty">Ford F-250</a> pickup-based, almost full lane-wide, 7 foot tall, 18 foot long monster. Basic everything inside. Geared and built for 1 thing – carrying a lot of heavy stuff from 1 place to another. Totally exceeded my requirements. I’d have been happy with a GMC Savana, or even just a minivan. But, it did have a line-in jack on the AM/FM radio.</p> <p>The reason we chose the Etobicoke Ikea was the covered parking. I hadn’t checked the dimensions on the van before entering the 9-foot limit parking garage, but it fit. It quickly became obvious that I’d need to park far from other cars because of the tight spaces and corners. Pro tip #2: when driving a new vehicle know it’s dimensions.</p> <p>Shopping was ok, but the play area was full when we went to drop our son off, so there was much crying. It was a little heartbreaking to have to tell him because we’d been mentioning it all day. Ikea is really best visited with children on a weekday.</p> <p>Ikea has spots near the door for loading your car, but in the covered parking you cannot back an 18 foot van into one of those spots. Pro tip #3: stick to the uncovered parking with anything bigger than a minivan. Or just wheel your purchases to your present, easily accessed parking spot.</p> <p>Furniture loaded, we headed home. Do not try to drive in anything but the right lane on the highway in a Ford E-250. It really shouldn’t go above 110 km/h. Anything more is a little hard to control. It’s so wide that even the smallest variation can leave you too close to the next lane. That’s how it felt, anyway.</p> <p>By the time it was time to take the van back both children were fast asleep. This meant I’d need another cab ride home. Add another $25. Replacing gas cost $20.</p> <p>The final total for the day was $83.62 for U-Haul + $50 for 2 cab rides + $20 for gas = $153.62. We saved $15.38 over Ikea’s deliver cost. On the plus side, we got the furniture on the same day.</p> <p>This could be done for much less than Ikea’s delivery cost. Eliminate the need for the taxi, research insurance coverage, won’t need the dolly next time. Renting a smaller vehicle from a car rental place would probably help, too. Enterprise rents a GMC Savana/Express for $50, which includes 200 km. They’d probably pick you up, too.</p> <p>By the way, I didn’t even need the dolly. The thing I wanted to move was too big for it, and it came apart just like Ikea furniture. Live and learn!</p> James McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.com0tag:blogger.com,1999:blog-6792849853700107540.post-55097728772683820782012-04-26T23:35:00.001-04:002012-04-26T23:35:13.309-04:00A Wizard Should Know Better<p><a href="http://www.youtube.com/watch?v=v7NzBTRzCkg">http://www.youtube.com/watch?v=v7NzBTRzCkg</a></p> <p>I recently quit at CIBC and started working at Evault. This means that my wife and I lost our immunity to account fees, kind of like when Superman transformed into a normal human. We already had a couple of President’s Choice Financial accounts so I decided to work towards closing the CIBC ones and use only the PCF accounts. It’s been a lot more complicated than I thought. And the reason should embarrass any developer: our accounts have dependencies.</p> <p>In programming, one aims to create a system where no one part knows anything about how other parts work. This is so that you can change one part without other parts caring. Changes are limited to individual parts and don’t ripple through the system. Thus you create a system with few, if any, dependencies on parts working a certain way.</p> <p>You know how it goes with banking, though.. Sure, employer, direct deposit my paycheque. Sure, insurance company, direct debit my premiums. Sure, bank, direct debit my mortgage. Convenience now, pain later as the dependencies pile up.</p> <p>Some of these are probably unavoidable, and worth the hassle, like the mortgage payment, the paycheque and withdrawals for investing. But that’s about it. We really should be paying everything else as a bill monthly. Companies don’t care where the money comes from, just that account 12345 is paid in full.</p> <p>The dependencies can get created recursively on you, too. Your employer passes on the account info to your benefits and group RRSP provider(s), so you have to update those. Hopefully these are provided by the same company, and can be changed online. Otherwise you have to call someone, and possibly send in a void cheque.</p> <p>This money stuff is all virtual. There is no <em>technical</em> reason for any of this to involve talking to humans on the phone, in person, or sending snail mail. The reason that these settings typically aren’t available online is that the systems that process this stuff within and between companies is very complex. This is because it’s been built up over 30+ years. Electronic payment systems predate the web by a large margin.</p> <p>Our transition from CIBC to PCF is so complex that I created a <a href="http://trello.com">Trello</a> (task management) board. As I write this we’re at 10 tasks done, 7 pending. Until we complete the process we have to auto-transfer money from CIBC to PCF so that there’s money for the insurance, mortgage, etc.. Programmers might call it a bridge.</p> <p>The thing that makes this transition process possible is PCF’s ability to link accounts at other banks to PCF accounts, allowing you to transfer to and from PCF accounts. I’m pretty sure this is available at other discount banks, like ING. Why not at the main banks? Not in their interest, plus it’s technically complex (though not impossible.) This would be much, much harder if the switch were from CIBC to, say RBC. There’s no way to pay your CIBC line of credit from an RBC account online. You’d be forced to transfer your line of credit to RBC, or visit a CIBC branch once a month with an RBC cheque.</p> <p>So, do yourself a favour and limit the dependencies on your big-5-banks accounts to paycheques, strict-deadline payments like mortgages, and accounts at other banks between which you transfer money, like investment accounts. And transfer to a low cost bank like PCF.</p> <p>Keep your accounts agile.</p> James McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.com0tag:blogger.com,1999:blog-6792849853700107540.post-87464008742561545052012-04-26T20:58:00.001-04:002012-04-26T20:58:23.895-04:00The Truth Come Out<p>A few days ago:</p> <p>Me: “Should we have some of [3 year old] Ethan’s Easter chocolate?</p> <p>Wife: with apparent shock, “Oh, no, that’s his chocolate. If we want some we should just go to Mac’s.”</p> <p>Me: “Ok, I’ll just have some fruit.”</p> <p>Tonight:</p> <p>Me: digging through the cupboard to find an empty Hershey’s Snapsy chocolate bunny box: “Mummy, what happened to Snapsy?”</p> <p>Wife: sheepish grin.</p> <p>Me: “His chocolate, eh? At least he still has this other chocolate bunny.”</p> James McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.com0tag:blogger.com,1999:blog-6792849853700107540.post-25608751589878266882012-02-14T00:47:00.001-05:002012-02-14T00:47:20.263-05:00Unit Testing HTML Parsing, While Keeping AppHarbor and RefactorPro Happy<p> </p> <h3>Short Version</h3> <p>“Unit” tests were starting development web server, reading web page responses using HttpWebRequests, loading them into HtmlAgilityPack HtmlDocuments and parsing them. AppHarbor (it’s American, so no u after o) wouldn’t start the dev web server and RefactorPro parsed a very large test html page because it was part of a VS project, which took up gobs of memory. The solution was to turn the html pages into txt files, store them as project resources and use HtmlDocument.LoadHtml to keep everything self contained.</p> <h3>Long Version</h3> <p>Yes, my unit tests were actually starting a web server and reading test versions of pages from it. This breaks the general tenant of unit testing that they shouldn’t go “out of memory”, meaning don’t read from disk, database, network etc. Everything should be contained within code.</p> <p>So, it should have been no surprise that AppHarbor wouldn’t start a copy of the development web server for my unit tests to run against. I knew the time had come to somehow bypass the network and read the HTML files from disk. Use of the local dev server would be limited to working offline on the GO train.</p> <p>I was also facing an issue with DevExpress’s RefactorPro. It likes to parse every HTML file in your Visual Studio projects, just like it parses your code. Unfortunately, one of the test pages is a list of all 11,000 products at the LCBO, and the LCBO website’s HTML is needlessly complex. When the HTML page was included in the project it sent Visual Studio’s memory use through the roof. I like RefactorPro/CodeRush too much to keep it unloaded when working in my LCBO Drink Locator code to disable it, so I kept the HTML page excluded from the project. It was still on disk, and the dev web server would serve it up when requested anyway.</p> <p>My first approach to the AppHarbor issue was to attempt loading the existing HTML file from disk. My assumption was that if my unit test is in some bin\debug or bin\release folder then my HTML file is in ..\..\..\AnotherProject\doc.html. This is true on my PC, but on AppHarbor’s build server of unknown technology configured with unknown settings, subject to change, it was not so.</p> <p>The next thing I tried was to add the HTML doc as a project resource to one of the unit tests, and to load it into an HtmlAgilityPack HtmlDocument. Thank you, StackOverflow for suggesting the approach. Adding the HTML doc as a resource meant it was part of a project again, sending memory use sky high. Being a resource, I couldn’t just exclude it and still have it be read from disk. I could now load the doc from “memory” i.e. not from disk or network, but suffered long waits as RefactorPro parsed the LCBO’s crazy HTML. (How many nested tables does a page need?)</p> <p>I noticed HtmlDocument has a method called LoadHtml that accepts a plain string, which gave me an aha moment. There was no need to make the project resources actual HTML docs; they could exist as txt files. This meant RefactorPro would ignore them, and I could still load them from memory. A rename and re-add as resource fixed the issue.</p> <p>All my unit tests now pass on my PC and when AppHarbor runs them.</p> <p><a href="http://lh5.ggpht.com/-P7WeiShM8NU/Tzn1SIRLRqI/AAAAAAAAAH4/HqqJjb_qN0c/s1600-h/image%25255B8%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-c7HWpOu11-k/Tzn1S-13UjI/AAAAAAAAAIA/2l88FA6Dn-I/image_thumb%25255B4%25255D.png?imgmax=800" width="528" height="484"></a></p> <p> </p> <p><a href="http://lh3.ggpht.com/-iXoCDwBXe5A/Tzn1TQdIsTI/AAAAAAAAAII/ZdD_OOCk7xU/s1600-h/image%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-e5LoR_2MlW0/Tzn1UYp9dwI/AAAAAAAAAIQ/U285VUQ3l7g/image_thumb%25255B2%25255D.png?imgmax=800" width="644" height="480"></a></p> <p>Not sure why AppHarbor only sees 51 tests, and not 61, but ok.</p> <p>FYI, the app is hosted at <a href="http://lcbodrinklocator.apphb.com/">http://lcbodrinklocator.apphb.com/</a>. It’s a work in progress, but it’s ability to <strike>destroy planets</strike> find drinks is fully functional.</p> <p><a href="http://lh3.ggpht.com/-cP5pFFDau80/Tzn1Xe848_I/AAAAAAAAAIY/DBe0bS0-gBc/s1600-h/image%25255B12%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh6.ggpht.com/-v7DFFB7af6o/Tzn1Zqk-eBI/AAAAAAAAAIg/fgBIlFKss4Q/image_thumb%25255B6%25255D.png?imgmax=800" width="612" height="484"></a></p> James McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.com1tag:blogger.com,1999:blog-6792849853700107540.post-18111864173284815742012-01-19T14:22:00.001-05:002012-01-19T14:22:07.179-05:00Not All RRSP Contributions Are Equal<p>Ah, January, when thoughts turn to taxes. I still do my own because my situation is pretty simple. My yearly routine is to put on Ace Ventura Pet Detective and fill out forms for me and my wife. Since switching away from paper forms a few years ago to the fabulous and free <a href="http://www.studiotax.com">StudioTax</a> I’ve been able to watch more of the movie. StudioTax also has a special place in my heart because it’s written in .NET. I’m guessing the name comes from the development tool .NET folks use, Visual Studio.</p> <p>This post isn’t about StudioTax, though. It’s about something I’m sure I’ve understood from newspapers and magazines turning out to be wrong: that your entire RRSP contribution results in a rebate equal to your highest tax rate. This is the reason tax planners sometimes recommend not contributing to an RRSP until you are in a high tax bracket, like the last 2 federal ones. You don’t want to waste getting about 40% back by contributing when you’ll only get about 32% back. The advice is to set aside the money until you reach one of these brackets, just not in an RRSP. These days that place would be a TFSA account.</p> <p>It’s not so simple, though, as waiting until your total income moves you to a higher tax bracket and then plowing all of your savings into an RRSP account that year.</p> <h4>The Math</h4> <p>Disclaimer: I am not a tax planner, accountant or financial consultant. There are lots of other considerations than those below to make when deciding what’s right for your situation. I’m just exploring something I came to realize recently, which applies to my situation.</p> <p>The Canadian income tax system is what’s called progressive. Not progressive in the “this is the better way” sense, but in the “progression from one level to another” sense. Your entire income isn’t taxed at the same rate. The first $x is taxed at a certain level, the next $y is taxed at a higher level, the next $z at an even higher level. The feds have their own set of x, y, z values and rates, as does each province. Except Alberta where there is no provincial income tax.</p> <p>I’ll use small, hypothetical amounts for this discussion, and assume there are no automatic tax credits, as there are in the real tax system.</p> <p>Let’s say the brackets and rates are:</p> <table border="0" cellspacing="0" cellpadding="2" width="400"> <tbody> <tr> <th valign="top" width="200">Income</th> <th valign="top" width="200">Rate</td></th> <tr> <td valign="top" width="200">First $50</td> <td valign="top" width="200">15%</td></tr> <tr> <td valign="top" width="200">Next $50</td> <td valign="top" width="200">25%</td></tr></tbody></table> <p>You make $60 a year. That means you owe:</p> <table border="0" cellspacing="0" cellpadding="2" width="401"> <tbody> <tr> <th valign="top" width="137">Income</th> <th valign="top" width="136">Tax Calculation</th> <th valign="top" width="126">Amount</td></th> <tr> <td valign="top" width="137">On the first $50</td> <td valign="top" width="136">$50 x 0.15</td> <td valign="top" width="126">$7.50</td></tr> <tr> <td valign="top" width="137">On the next $10</td> <td valign="top" width="150">$10 x 0.25</td> <td valign="top" width="157">$2.50</td></tr> <tr> <td valign="top" width="137"> </td> <td valign="top" width="150">Total</td> <td valign="top" width="157">$10.00</td></tr></tbody></table> <p>Your employer would have taken this from your pay cheque, so you’ve already paid. If you don’t do anything more to do with taxes you owe nothing and you’ll get nothing back.</p> <p>Every dollar you contribute to an RRSP (or pension plan for that matter) lowers your taxable income by 1 dollar. Let’s do the same calculation but with a contribution of $4.</p> <table border="0" cellspacing="0" cellpadding="2" width="400"> <tbody> <tr> <td valign="top" width="200">Total Income</td> <td valign="top" width="200">$60</td></tr> <tr> <td valign="top" width="200">RRSP Contribution</td> <td valign="top" width="200">$4</td></tr> <tr> <td valign="top" width="200">Taxable income</td> <td valign="top" width="200">$60 – $4 = $56</td></tr></tbody></table> <p>Plug $56 into the tax calculation table and you get:</p> <table border="0" cellspacing="0" cellpadding="2" width="400"> <tbody> <tr> <th valign="top" width="133">Income</th> <th valign="top" width="133">Tax Calculation</th> <th valign="top" width="133">Amount</th></tr> <tr> <td valign="top" width="133">On the first $50</td> <td valign="top" width="133">$50 x 0.15</td> <td valign="top" width="133">$7.50</td></tr> <tr> <td valign="top" width="133">On the next $6</td> <td valign="top" width="133">$6 x 0.25</td> <td valign="top" width="133">$1.50</td></tr> <tr> <td valign="top" width="133"> </td> <td valign="top" width="133">Total</td> <td valign="top" width="133">$9.00</td></tr></tbody></table> <p>You’ve paid $10 through your pay cheque, you owe $9, so you get a rebate of $1. This is equal to your contribution times the higher rate, or $4 x 0.25 = $1.</p> <p>However, if your contribution lowers your income into the next bracket down, the entire contribution is not worth 25%. Only the portion that lowers your income to the edge of the higher bracket is worth the higher bracket’s rate. The remaining portion’s rebate rate is the rate of the lower bracket.</p> <p>To demonstrate, lets say you contribute $15 of your $60 income.</p> <table border="0" cellspacing="0" cellpadding="2" width="402"> <tbody> <tr> <td valign="top" width="200">Total Income</td> <td valign="top" width="200">$60</td></tr> <tr> <td valign="top" width="200">RRSP Contribution</td> <td valign="top" width="200">$15</td></tr> <tr> <td valign="top" width="200">Taxable income</td> <td valign="top" width="200">$60 – $15 = $45</td></tr></tbody></table> <p>Now you owe:</p> <table border="0" cellspacing="0" cellpadding="2" width="401"> <tbody> <tr> <th valign="top" width="133">Income</th> <th valign="top" width="133">Tax Calculation</th> <th valign="top" width="133">Amount</th></tr> <tr> <td valign="top" width="133">On the first $45</td> <td valign="top" width="133">$45 x 0.15</td> <td valign="top" width="133">$6.75</td></tr> <tr> <td valign="top" width="133"> </td> <td valign="top" width="133">Total</td> <td valign="top" width="133">$6.75</td></tr></tbody></table> <p>You’ve paid $10, you owe $6.75, so you’ll get a rebate of $3.25. Note that this is not equal to your contribution of $15 times the higher rate, or $15 x 0.25 = $3.75. This is because it’s a combination of the potion that brought your income down to $50 at the high rate, plus the remaining portion at the low rate. That is:</p> <table border="0" cellspacing="0" cellpadding="2" width="400"> <tbody> <tr> <th valign="top" width="133">Contribution</th> <th valign="top" width="133">Rebate Calculation</th> <th valign="top" width="133">Amount</th></tr> <tr> <td valign="top" width="133">Portion to get to $50</td> <td valign="top" width="133">$10 x 0.25</td> <td valign="top" width="133">$2.50</td></tr> <tr> <td valign="top" width="133">Remaining portion</td> <td valign="top" width="133">$5 x 0.15</td> <td valign="top" width="133">$0.75</td></tr> <tr> <td valign="top" width="133"> </td> <td valign="top" width="133">Total</td> <td valign="top" width="133">$3.25</td></tr></tbody></table> <h4>Conclusion</h4> <p>If you’ve been waiting to make contributions until you’re in a higher bracket, and you’ve finally reached that bracket, it’s not really worth contributing more than is enough to bring your income down into the next bracket. In the example above, it’s not worth contributing more than $10 of the $15 to bring your income down to the lower rate. Any more contributions will only earn a rebate of 15%. You’re better to hold on to any other savings to use in future years in hopes that your income rises faster than inflation.</p> James McLachlanhttp://www.blogger.com/profile/03556851142178139257noreply@blogger.com0