It is easy in mathematics teaching to focus on the "what" questions and miss "why" and "what if" questions. There is no doubt that one needs to develop facility and ease with the basic skills of mathematics so that on is not always stumbling over small matters, but the richness and true utility of the subject comes from its conceptual structure. Mathematical thinking should promote life skills. The curriculum should be the vehicle for learning, not itself the goal of learning. Mankind has not engaged in mathematics for thousands of years simply because it is useful, but because it opens to something more transcendental. Why do math? Because it is beautiful!
In order to promote true conceptual understanding, one can ask "meta-questions." These come in a number of styles: Spot the error, head-on approach, explain, think before you leap, discover and explore, jolt, and pushing the boundaries. Here are some examples to show you these questioning strategies that you can use as assessments that promote true understanding. These are just a few of the ones that I chose, otherwise it would be 5 pages long and no one wants to see that. Here is the pdf so you can view all of the resources: http://www.jamestanton.com/wp-content/uploads/2011/09/Assessment-Thoughts_APRIL-2012-Version.pdf