Thursday, December 25, 2014

User-Facing Exceptions

User-Facing Exceptions

After Mark Seeman’s recent blog post, Exception Messages are for Programmers, Mogens Heller Grabe commented:

I usually introduce a special exception when I’m building software for clients: a DomainException! As you might have guessed, it’s a special exception that is meant to be consumed by humanoids, i.e. the end user. This means that it is crafted to be displayed without a stack trace, providing all the necessary context through its Message alone. This is the exception that the domain model may throw from deep down, telling the user intimate details about a failed validation that could only be checked at that time and place.

I totally understand Mogens’s reasoning here. An exception’s primary goal is to halt execution in an error state, but there are some types of exceptions that really want to provide a user-facing message at the same time. They are still exceptional, but they know something about the nature of the error that would be useful for the user to be aware of.

However, there are some problems with Mogens’s strategy:

  1. Using a specific exception type (e.g. DomainException) means that you can’t use different exception types based on the nature of the exception, or other information that you might want to associate with it. What if I want a WorkflowDoesNotExistException in one case, and a ServiceNotAvailableException in another, but I still want to provide a user-facing message with each?
  2. As Mark Seeman very powerfully argues, exception messages are for programmers. Their purpose is to provide enough information for developers to know what went wrong, how to reproduce the issue, and how to fix it. The exception message is included with the stack trace when you call ToString() on the exception. Co-opting the exception message to provide user-facing information gets in the way of its primary purpose.
  3. It’s good practice to catch exceptions and use them as the InnerException to a new exception with additional debug information, at various levels of the call stack. You can’t just catch DomainExceptions and treat them differently from your other exceptions, because they could be thrown at any level of your application, and subsequently wrapped in other exceptions.

IUserFriendlyException

Here’s my solution to the problems mentioned above. I’ve been using it for a couple of years now without any problems.

public interface IUserFriendlyException
{
    string UserFacingMessage { get; }
}

This solves the first problem by using an interface, which can be implemented by any number of exception types.

It solves the second problem by using a new property–UserFacingMessage–which can be provided in addition to (not instead of) your exception’s programmer-targeted Message. Your programmers still get the debug information they want, and users aren’t exposed to technical details they don’t care about.

A Note Regarding Internationalization: While I haven’t personally had to worry about producing different user-facing messages for different users, Mark Seeman correctly points out that this is an important issue to consider in many applications. Fortunately, you can use this same pattern for projects that use the ResourceManager to produce culture-specific strings.

public interface IInternationalUserFriendlyException
{
    string UserFacingMessageResourceKey { get; }
}

But what about the third problem? This requires a little more work. It’s a good idea to catch and log all exceptions at the Presentation Layer of your program, so that users don’t get ugly server error pages, or crashing applications. You don’t care what type of exception it is, and you typically don’t have anything specific to say to the user at this point.

private static readonly string GenericErrorMessage = 
    "We are really sorry, but something unexpected happened. " +
    "Please refresh the page and try again. " + 
    "If the problem persists, please report it to your administrator.";

public void OnException(ExceptionContext filterContext)
{
    Log.Error(filterContext.Exception);
    ShowError(filterContext, GenericErrorMessage);
    filterContext.ExceptionHandled = true;
}

But now if you have a user-facing message, you can provide that message instead.

public void OnException(ExceptionContext filterContext)
{
    Log.Error(filterContext.Exception);
    ShowError(filterContext, 
        filterContext.Exception
            .AndInnerExceptions()
            .OfType<IUserFriendlyException>()
            .Select(e => e.UserFacingMessage)
            .FirstOrDefault()
            ?? GenericErrorMessage);
    filterContext.ExceptionHandled = true;
}

The only tricky part is that you have to dig deeper than the top-level message. The code above uses this extension method:

public static IEnumerable<Exception> AndInnerExceptions(
    this Exception exception)
{
    while (exception != null)
    {
        yield return exception;
        exception = exception.InnerException;
    }
}

Conclusion

I’ve found this strategy to work well in those cases where an exception wants to communicate user-facing information. Hopefully you will, too. Do you have questions or comments? Leave them in the comments section below.

Saturday, November 01, 2014

Ten Things Beginners Must Know About JavaScript

Introduction

No doubt about it–JavaScript is absolutely essential for any web developer. Gone are the days when you could get by with a little CSS and HTML. Fortunately, it has so few rules that it’s dead simple to get started:
alert("Hello World."); // Shows the user "Hello World." in a dialog
That’s it! Many simple tasks can be accomplished with just a few lines of code. There’s no need to import packages, or declare namespaces. You write some code, and it runs. However, the very lack of structure that gives JavaScript its low barrier to entry also enables beginning developers to write unstructured, fragile code without realizing it. As an application grows, this unstructured code will come back to haunt you in the form of unexpected, difficult-to-find bugs.
In this tutorial, I plan to correct some of the common misconceptions and mistakes that cause undue pain for beginning JavaScript developers. Here are a few things every beginning JavaScript developer should know:

1. You can add properties to almost everything

JavaScript has only three primitive data types: String, Number, and Boolean. Everything else (if it’s not null or undefined) can have properties added to it.
Note: Even though String is a primitive type ("hi"), there is another incarnation of String in JavaScript which is an object (new String("hi")). See here for details.
You may have seen something like this:
var a = {}; // create a new object
a.b = 'hello';
In the above code, a.b meant nothing until I set a value to it. Now a has a property called b. But wait, it gets weirder.
var a = []; // create an array
a.b = 'hello';
Great, so now you’ve got an empty array, with a property on it called b.
Caution: Just because you can put properties on arrays doesn’t mean you should. As I’ll discuss later, the above code will change the behavior of for...in loops on that array.
How about this?
var a = function() {};
a.b = 'hello';
Yes, that’s right. a is now a function, with a property. Which brings me to my next point.

2. Functions are objects

More and more languages are getting support for treating functions as values, but depending on your background this may not be at all familiar to you.
function doIfTrue(isTrue, whatToDo)
{
    if(isTrue)
        whatToDo();
}
doIfTrue(true, function() {alert('hello');}); //  alerts "world"
doIfTrue(false, function() {alert('world');}); // does nothing
The function above treats the whatToDo parameter as a function. This sort of pattern allows developers to do some very powerful things like setting up event handlers with very little code.

3. for...in loops iterate over property names, not values

Developers coming from a Java or C# background are used to so-called “foreach” loops, which iterate over all the values in a collection. JavaScript doesn’t (currently) have an equivalent loop. The closest thing, a “for in” loop, has some important differences:
var arr = ['a', 'b', 'c'];
for(var i in arr) {
    alert(i);      // 0, 1, 2
    alert(arr[i]); // 'a', 'b', 'c'
}
As you see, the variable used by the for...in loop gives you the keys by which other values can be found, but you have to perform the extra step of getting the values off of the original object.
Why does JavaScript do this? The main reason is that for...in was not made for arrays: it was made for objects that have properties on them:
    var pairs = {'a': 'apple', 'b': 'banana'};
    for(var key in pairs) {
        alert(key);        // 'a', 'b'
        alert(pairs[key]); // 'apple', 'banana'
    }
Caution: Because a for...in loop iterates over the properties on an object, you’ll get strange behavior if you use the loop on an array that has extra properties added to it, For this reason, you should avoid using for...in loops on arrays–use simple for loops instead. They’re faster anyway.
Note: ECMAScript 6 will introduce for...of loops, which iterate directly over values.

4. Variable scoping

All developers, in every language, should avoid using global variables. But it’s easy to do by accident in Javascript because nobody’s forcing you to organize your code into modules.
var message = "hello world";
alert(message);
If the above code is run all by itself in the browser, it will create a new global property on the window object, called message. That means that if any other code on your site also has a similarly-declared variable called message, they’ll stomp on each other.
In Javascript, all variables declared with the var keyword are scoped to their declaring function. That means if you want to make sure your variables aren’t global, you should put them in a function, and then call that function.
(function() {
    var message = "hello world";
    alert(message);
})(); 
Developers with a background in other languages based on C syntax are used to variables scoped by curly braces ({}). In JavaScript, variables declared with var are “hoisted” to the top of the function they’re declared in. This code:
function sayHi() {
    if(true) {
        var s = "hi";
    }
    alert(s); // alert("hi") -- `s` is still within scope.
} 
… is the same as this:
function sayHi() {
    var s;
    if(true) {
        s = "hi";
    }
    alert(s);
} 
This also means that if you refer to a variable outside the scope of your function, you’ll use the value of that variable at the moment the code is run, not at the moment your function is created. This trips up beginners all the time:
 var thingsToDo = [];
 for(var i = 0; i < 2; i++) {
     thingsToDo.push(function() {alert('hello ' + i);}); 
 }
 for(var k in thingsToDo) {
     thingsToDo[k]();               // alerts "hello 2" twice.
 }
Remember that the variable i starts as a 0 value, but by the time it’s called (in the second loop) its value has been incremented to 2. To capture the value of i when you create the function, you have to create another function scope:
var thingsToDo = [];
function createHelloAlert(v) {
    // `v` only exists within this method call.
    return function() {alert('hello ' + v;} 
}
for(var i = 0; i < 2; i++) {
    thingsToDo.push(createHelloAlert(i)); 
}
for(var k in thingsToDo) {
    thingsToDo[k]();  // alerts "hello 0", then "hello 1".
}
Note: ECMAScript 6 will introduce a let keyword, which will allow you to declare a variable scoped to the curly-braces. This is known as Lexical Scoping.

5. Variables that aren’t explicitly declared can be global

Suppose you remember to wrap your code in a function, but forgot the var keyword:
(function() {
    message = "hello world";
    alert(message);
})(); 
When you set a variable’s value, and you haven’t declared it to be a variable for the current function scope via the var keyword, JavaScript assumes you mean this.propertyName. So the above code is the same as this:
(function() {
    this.message = "hello world";
    alert(this.message);
})(); 
If you’re a Java developer, you’re saying to yourself, “What’s this? No, really. What is this?” In other languages, this means the instance of the class that you’re looking at, but in JavaScript it means the object that your function got called on. If you’d done this:
var a = {
    foo: function() {
        this.message = "hello world";
    }
};
a.foo(); // `a.foo()` means foo's `this` is `a`.
alert(a.message); // outputs "hello world"
… then calling the foo method puts a value on a’s message property. But since our original code snippet is calling a function that doesn’t have anything in front of it, this is given a fallback value of window. You end up creating another global variable just by forgetting the var keyword.

6. Understand how .prototype works

JavaScript uses a special prototype property to solve the problems that other languages use classes to solve. Consider the following:
function Person(first, last)
{
    this.first = first;
    this.last = last;
}
var john = new Person("John", "Doe");
var mary = new Person("Mary", "Deer");
Person.prototype.full = function() {return this.first + " " + this.last;};
alert(john.full());
There are a lot of things happening here.
  1. We create a function, which will set properties on its this object when called.
  2. We create two separate instances of that function by putting the new keyword before our function calls. This ensures that john and mary refer to completely separate objects, each with their own first and last properties.
  3. We create a new function and assign it to the full property on our Person function’s prototype property. The prototype property exists on all functions, and allows you to define fall-back properties that should exist on every object created from that function.
  4. We call the full() function on john. JavaScript sees that the john object doesn’t actually have a full function on it, so it looks for a Person.prototype.full() function and calls that instead. Within that call, however, this still refers to the john object.

7. JavaScript never sleep()s

Many languages support the notion of threads, which allow you to have several procedures running at the same time. The problem is that multi-threading opens a huge can of worms: thread locks and other race conditions that even the best developers have a hard time grappling with.
JavaScript avoids these issues entirely by only allowing one piece of code to run at a time. This is great, but it requires us to write our code in a different way. Most languages create a layer of abstraction over operations where the program is waiting for something to happen. For example, in Java, you can call Thread.sleep(100), and the rest of your procedure won’t run until 100 milliseconds have passed.
When you’ve only got one thread to work with, you can’t afford to have it doing nothing for hundreds of milliseconds at a time–it would freeze up the UI and make a horrible user experience. So JavaScript almost never blocks execution. Instead it relies on asynchronous callbacks to let you say what should happen when an operation finishes. For example:
window.setTimeout(function() { console.log(a); }, 1000);
console.log('hello world');
var a = 'got here';
The above code will first create a “hello world” message (even though that line of code comes after the setTimeout() method), and then print “got here” one second later. Even if we changed the timeout to 0, we know that the a variable will be initialized before getting logged because no other JavaScript can run until this code has finished.
Beginners will often write code like this, and then wonder why a doesn’t have a value:
var a;
$.ajax(url, {success: function(e, data) { a = data; });
console.log(a); // BAD! This line will run before `a` gets set to a value!
Don’t fall into that trap.
There are a few caveats that I should mention.
  1. Not all functions that you pass into other functions are asynchronous callbacks. The following code works just fine, because doIfTrue() calls its whatToDo argument, synchronously, before it returns:
    var a = 'hello';
    doIfTrue(true, function() {a = 'world';});
    console.log(a); // prints "world"
    
  2. alert() is one of very few exceptions to the no-blocking rule–nothing happens until the alert window closes. Even the timeouts freeze! This is one reason that it’s usually best to avoid using alert().
  3. Web Workers can allow you to run CPU-intensive tasks on a separate thread, but they’re structured very carefully to avoid race conditions. Beginning developers rarely need to worry about such things.

8. Automatic type conversions

Like many languages JavaScript does some automatic conversions between types under certain circumstances. For example:
var s = 1 + ""; // yields "1" 
JavaScript takes this a little further than many languages, though, and you can leverage this fact to make your code very concise. For example, instead of if(a != null) {...}, you can just say if(a) {...}. Even though a isn’t a Boolean, JavaScript can tell you want to treat it like one, so it determines whether a’s value is “truthy” or “falsy” and acted accordingly.
Falsy JavaScript values include:
  • false
  • 0
  • empty strings ('' or "")
  • null
  • undefined
  • NaN
Everything else is Truthy.
Here’s another example:
var a = b || c;
If b has a truthy value like “Bob”, that’s the value that a will get. If b is falsy, a will get the value of c.
You can force a truthy/falsy value to become a true/false value by negating it twice:
var trueOrFalse = !!value;
Also, in equality comparison, x == y has some complex rules for converting different value types. This can yield weird behavior, like:
var a = "1" == true; // true: "1" -> 1, which is truthy
And you can use the === operator for strict comparison without automatic conversion.
var a = "1" === true; // false: different-typed values are not equal.

9. JavaScript is code–not data or markup

For simplicity, many online tutorials will mix JavaScript and HTML, like so:

Mixed HTML and JavaScript (bad)

<button onclick="return confirm('Delete this object?');">Delete</button>
This works fine for small tutorials, but it has some serious drawbacks when you’re writing real applications. It mixes the programmatic behavior of the UI elements (represented by JavaScript) into the structure and data that is (represented by HTML). HTML should not be deciding how it interacts with the JavaScript. Instead, separate your JavaScript code from the HTML, and let it decide how it interacts with the page.

HTML

<button data-confirmation="Delete this object?">Delete</button> 

JavaScript (with jQuery)

// Set a click handler for anything with a data-confirmation attribute.
$('[data-confirmation]').click(function() {
    var message = $(this).data('confirmation');
    return confirm(message);
});
As you can see, the JavaScript has now defined the behavior for any element with a confirmation data attribute. This approach, sometimes called “unobtrusive JavaScript,” has several advantages, including:
  1. It usually makes HTML code more concise and readable.
  2. It allows you to declare a UI concept in one place and reuse it throughout your system.
  3. If you decide to change your implementation details (e.g. “browser confirmation dialogs are ugly and inconsistent–we want to use a popup instead!”), you can do this in one place without touching your HTML.
If you’re rendering templates in the browser à la Angular or Polymer, this separation means putting HTML in one part of your file and JavaScript in another. If you’re using a server-side engine like JSP or Razor, you’re better off keeping the JavaScript code in static, cached .js files, away from all your dynamically-generated HTML tags.

10. JavaScript is not just for browsers

JavaScript has clearly come a long way since it was created (purportedly in ten days) in 1995. Now, technologies like node.js allow JavaScript to be run outside of any browser. Some common use cases for JavaScript as a general scripting language include:
  1. Writing server-side code in JavaScript. Ghost.org is just one example of a web application whose server-side code is JavaScript running on node.js.
  2. Building LESS files into CSS. Less.js is the fastest, most accurate LESS converter around, and it’s written in JavaScript. If you use LESS, but don’t want your LESS to get re-translated client-side on every page load, it’s a good idea to pre-build the CSS by invoking the lessc compiler via node.js.
  3. Managing build tasks. What if you want to run unit tests against your JavaScript every time you change a file, to make sure you didn’t break anything? Grunt.js, with its watch and Karma plugins can do that! Or you can configure Grunt to rebuild all those CSS files every time you change a LESS file.
If you’re developing web applications, you’re going to need to learn JavaScript. You might as well apply that knowledge to other aspects of your application too!

Conclusion

Like it or hate it–JavaScript is here to stay. It is currently the de-facto language for building any kind of rich user experience on the web.
JavaScript is powerful, and full of intricacies and nuances. The more you learn about JavaScript, the more you realize how little you actually know about it. But arming yourself with knowledge will help you avoid costly mistakes that are common for beginning developers.
Good luck.

About the Author

I’m a Software Architect with over 7 years of experience in building dynamic web applications. I have a passion for learning, and for helping other people learn. While earning over 60,000 reputation points on StackOverflow, I’ve seen a lot of the simple mistakes that vex JavaScript developers on a regular basis. If you have questions about JavaScript, or programming generally, feel free to reach out to me at https://www.codementor.io/j2jensen
Special thanks to Aaron Frost and the peer reviewers at CodeMentor for their valuable feedback as I crafted this article.

Saturday, September 27, 2014

Provident Funding: Providing you with a false sense of security

Update: 12/28/2016

I'm not sure when it happened, but I noticed that Provident Funding now supports mint.com integrations! They also seem to have changed at least a couple of the other issues I complained about here. Yay!

Original Post

As a software developer for web applications, I take a keen interest in Internet Security. One of the more interesting aspects of this field is that there are some practices intended to make a site more secure, which don't always actually improve security. At best, these practices are an unnecessary burden to the user. At middling, they'll give the user and the provider a false sense of security, and make them less likely to notice other, more important issues. And at worst, they'll actually cause a user to compromise his security more by creating workarounds to byzantine policies.

A classic example is password strength rules. For those with enough training in information technology, this simple comic is enough to explain why the password strength rules used by most websites have trained most people to come up with passwords that are hard to remember, but easy to hack.

And in fact, the whole concept of a password is fundamentally flawed: every time you log in, you have to enter your password. That means that any time you use a computer that might have had a keylogger installed, or any time you enter it while someone might have been looking at your fingers, or a video camera might have caught your fingerstrokes, your password is potentially compromised. The very act of entering a password represents a security vulnerability in and of itself. We just haven't figured out a better solution that's convenient enough to work for most people.

I think the standard minimum password length for most websites I've seen recently has been 8 characters, but they insist on you mixing numbers, symbols, and upper- and lower-case letters. The problem is that most people choose ways of adding these elements that are dead simple for a hacker and his tools to guess. So they hardly add any difficulty at all if someone is trying to guess your password. At the same time, 8 characters isn't really enough to prevent the types of attacks that these rules are trying to prevent. This topic is worth an entire blog post of its own.

But as bad as that is, there are occasionally even worse cases. For example:

  1. Until a couple of years ago, American Express's website limited peoples' passwords to 8 letters. You couldn't create a longer, stronger password even if you wanted to!
  2. I once asked the company handling HR for an employer to send me my username, because they'd used an auto-assigned username that I could never seem to remember. A kind lady there sent me an email with both my username and my password in it. And this was the company handling my paychecks! This was at least three strikes against that company all in one go: 
    1. It implies that the company stores passwords in a way that it's possible to retrieve them. 
    2. It means that the people working for this company have the ability to see these passwords (not just have them automatically sent to users, but actually see them.)
    3. Email is not secure, and should never be used to send passwords (except possibly a temporary, random password that you're required to change within a time limit.)
When people managing a web application are making decisions about their security policies, they need to think very carefully about them. Even policies that seem like they'll make things more secure might encourage worse security practices. For example, if you make users change their password every few months, they're most likely going to do one of the following:
  1. Stop using a decently unique password that they would have remembered through muscle memory, and switch to using an easy-to-guess pattern, so they don't have to keep trying to think up a new one every three months. Variants of spring/summer/fall/winter are very common in this case.
  2. Keep using the same basic password, but change it in a predictable way. (e.g. add 1 to a number at the end every time they have to change it)
  3. Put their passwords on a sticky note next to their monitor, at least for the first week or two. (Many people do this anyway, but they'll be far more tempted if they're constantly being forced to come up with new passwords.)
Any time you introduce a procedure that gives the illusion of added security, without actually causing things to be more secure, you create a false sense of security, which can be dangerous. I'd like to highlight some of these false security procedures that are practiced by Provident Funding, a loan servicer:
  1. They stopped allowing users to connect their Mint.com accounts to their Provident Funding accounts. They claim that this is to improve the security of their customers because they don't have any control over what happens to that information once it enters Mint.com.
    1. This might be a valid concern for their customers, but not for the company itself. After all, the company doesn't have power over what users do with their own information that they view on their website either. There's nothing stopping those users from downloading all their statements and sending them to Nigerian con artists, if that's what they choose to do with their own data. 
    2. They used to have this connection to Mint.com. Are they trying to say that they were not secure before?
    3. Mint.com is owned by Intuit, who also provides such products as Quicken and TurboTax. Do you really think that their security practices are going to be anything less than impeccable?
    4. Most users don't actively manage their loan accounts from month to month. In other words, if they could see that they're payments are on track each month using a read-only service like Mint.com, they'd almost never have to actually log in to Provident's website. By forcing users to log in more often, Provident provides that many more opportunities for bad-guys to capture your password. If a bad-guy gets access to Mint.com credentials, they can see what a user spends their money on, but if they get access to  Provident credentials, they can do more useful things like change billing addresses and who-knows-what-else.
  2. Provident forces users to change their password every six months. As mentioned earlier, the practical value of this practice is questionable. But it truly becomes a false security practice when they allow users to reset their password to the same value as before. The site acts like it's got a security procedure, but all it really does is force a user to enter their password a bunch of times. Remember what I said earlier about the very act of entering your password? Yeah.
  3. When changing their password, the user is required to enter their username and password again. I understand requiring the password, but the username is prominently displayed at the top of the page, so asking people to enter it again is completely useless from a security perspective.
  4. Provident's password requirements are pretty close to the same as most websites, as mentioned above, except that the symbol character must be one of the following: !@#$-_. So rather than making the password harder to guess, this actually makes the hacker's job easier: he no longer has to worry that every character might be any symbol--he can now assume that one (and for 99% of users it'll be only one) of the password's characters is one of only seven possible values. 
Now, I appreciate that in some areas, they do adhere to some real best-practices. They don't send your statements to you in an email, for example. But when it comes to false security practices like those above, I have to wonder:
  1. Do they know that these practices are useless, but feel it's important to give users a sense of security just to keep up appearances? If so, that's really annoying and a little dangerous.
  2. Do they actually think that these practices have some value? If so, they're inept when it comes to real security, and we have to wonder what true vulnerabilities they left open while they followed these red herrings.
  3. Are some of these "security practices" signals that they have some really bad practices underlying their entire site, which they've had to work around? For example, are they failing to encode parameters, so they disallow funny characters in your password because they're afraid of little Bobby Tables? Are they blocking Mint.com because they have no confidence in their technical ability to keep an integration endpoint up and running? If so, we have to wonder whether they've got the technical competence to keep our data safe from real security threats.
I brought up many of these issues in an email directly to Provident months ago, and didn't get a very satisfactory response. Since there appears to be no sign of policy changes at this point, I'm hoping a little public shaming will get the attention of someone who cares. Feel free to share with people who are interested in this sort of thing.

Tuesday, September 09, 2014

The Big Fat Fraud

The other day, on Radio West, I heard part of an interview with Nina Teicholz, author of The Big Fat Surprise: Why Butter, Meat and Cheese Belong in a Healthy Diet. For a while, I was fascinated as she recounted various ways that the medical and scientific communities had latched onto ideas about fat and cholesterol, ignoring evidence contrary to those ideas.

The Big Fat Surprise

Teicholz's claim is basically that trying to reduce fat in our diet has had the opposite effect from what was intended. This concept is unsurprising to me. When trying too hard to avoid any one kind of food in our diet, it's easy to replace that thing with even less-wholesome alternatives. I remember a man in a birthing class trying to figure out a good diet for his wife, who was a "vegetarian." It turns out that while she didn't eat meat, she didn't eat any vegetables either, which basically left nothing but processed carb-rich foods on their menu. A recent study showed that a low-carb diet is actually twice as effective as a low-fat diet when trying to lose weight, adding to a mounting body of evidence that we need to stop making fat the bogeyman it has been for some time. In general, it's best to eat natural, whole foods, with as little processing as possible:  Butter is probably more healthy than margarine. Whole milk is probably more nutritious than skim milk or soy milk. Many of the things Teicholz was saying gibed with other things I'd learned. And she came across as very smart, knowledgeable, and convincing.

Pretty soon, though, the things she said stopped "ringing true." I'm not sure exactly when it was. It might have been when she started bashing on plant-based foods. "The evidence behind 'mostly plants,'" she said, "turns out to be quite thin." I may not be a nutritionist, but I'm passably familiar with nutrition science and plant-based diets in particular, and I can say with some certainty that there's a sizable body of evidence showing the benefits of eating fruits and vegetables.

Then a nagging suspicion started forming in the back of my mind as she shared her experiences with trying to set up interviews with some researchers:
I would get on the phone with researchers, and they would say, "If you're taking the Gary Taubes line, I won't even talk to you."
In my experience, when otherwise logical, well-educated people are completely unwilling to talk to someone, there's a reason behind it that's a little stronger than mere institutional bias. How did this "Gary Taubes" earn such a bad reputation in the scientific community? Did he interview researchers, and then take their statements out of context? Did he present the researchers' findings as supporting evidence for claims that they didn't actually support? Is he guilty of pseudoscience--the scientific community's equivalent of blasphemy? And if Teicholz is "taking the Gary Taubes line," then is she doing the same thing?

I remembered that some people will say what other people want to hear, because they know that other people will pay money to hear it. This is just a secular version of what the Book of Mormon calls "priestcraft." One website I stumbled upon claimed to have proof that all the health experts were wrong, and the best diet actually consists mostly of bacon and beer (no joke!), and if you send a check to such-and-such address, they'd send you more information about it. Was Teicholz's book just another incarnation of the "eat drink, and be merry, and everything will be okay" story that charlatans have been selling since time immemorial?

Just another Fad Diet Book

So when I got home, I did a simple Google search: "big fat surprise critical review." And lo, there it was: a huge two part article on a blog titled The Science of Nutrition, which tears The Big Fat Surprise to shreds. In summary:
What makes this particular book interesting is not so much that it is bad (which it is) or that it is extravagantly biased (which it also is). No, what really fascinates me about this book is that the author excessively and shamelessly lifts other people’s material. Most notably Teicholz lifts from another popular low-carb book called Good Calories, Bad Calories (GCBC) by Gary Taubes.
You probably don't have time to read through the whole thing--I didn't. But please go ahead and read a page or two, and then scroll to the bottom to see just how much content there is. You'll get a general idea of just how Nina Teicholz went about misinterpreting evidence, failing to find original sources, taking statements out of context, and so on. As the author concludes:
The issues I bring up in this review are too substantial and too numerous to be ignored. If you were to remove all of the instances where Teicholz deeply distorts a study or publication, and you were to remove all conclusions that she draws from the distortions you would be left with nothing but a pamphlet.
Every few years, it seems, a new book is published telling people about some simple change they can make that will help them lose weight and feel healthier. And every time a bunch of people rave about it, until they forget about it, and end up the same weight they were before. And the only people who really benefited were the author and publisher of the book. Meanwhile, scientific study after study confirms that the only way to consistently lose weight and keep it off is to do what experts have been saying all along: eat a variety of whole, fresh fruits and vegetables, limit how much food you eat, and get plenty of sleep and exercise. Anyone who tells you otherwise is selling something.

Some words of wisdom

Mormons believe in a code of health representing the "will of God in the temporal salvation of all saints in the last days." Revealed in 1833 to the prophet Joseph Smith, the Lord's pronouncement began:
In consequence of evils and designs which do and will exist in the hearts of conspiring men in the last days, I have warned you, and forewarn you, by giving unto you this word of wisdom by revelation
The Lord warned against the use of alcohol, tobacco, and stimulants. It emphasized a diet rich in grains and a variety of fresh fruits and vegetables, even encouraging abstinence from animal meat except when necessary.

When I became a member of the LDS church, I decided to follow this counsel more fully than most Mormons do--for about six years I was a vegetarian. I definitely experienced the blessings associated with this scripture:
 18 And all saints who remember to keep and do these sayings, walking in obedience to the commandments, shall receive health in their navel and marrow to their bones;
 19 And shall find wisdom and great treasures of knowledge, even hidden treasures;
 20 And shall run and not be weary, and shall walk and not faint.
I'm still very fit, despite sitting at a computer way more than I should. But during those years after my baptism I was more healthy than at any other time in my life, and there were several times when I was amazed at how well I could "run and not be weary."

Since 1833, modern prophets have clarified, expanded on, and re-emphasized portions of this Word of Wisdom. For example, the prohibition of tobacco and alcohol has been extended to include illegal substances that didn't exist in Joseph Smith's time, and obedience to the Word of Wisdom is now a requirement to be worthy to enter the Lord's temples. We are also encouraged to use our own understanding to help keep our bodies healthy--for example, many Mormons avoid all forms of caffeine, rather than just coffee and tea.

At the same time, science has increasingly found the basic dietary guidelines from the original revelation to be good, sound advice. When I hear advice that directly contradicts counsel given by the Lord through His prophets, I'm going to choose the Lord's way. In the end, I think the Lord's wisdom will always be found to trump the knowledge of man.