Monday, June 16, 2014

End-of-Chapter Questions and Oracle Certification Preparation

An Oracle exam candidate sent me the following question recently after reading my study guide for the 1Z0-117: Oracle Database 11g Release 2: SQL Tuning exam:

"One thing which I felt is that there should be mock Q&A's or Quiz to test the knowledge gained after finishing each chapter. Having these gives confidence and a practice before going for the exam."

This is a subject that I have a strong opinion about. I thought it deserved a detailed response rather than a brief email. I will state up front that my study guides will never have practice questions in them for several reasons.

The first problem with adding end-of-chapter questions is economic.  My guides present the information on exam topics in a very concise fashion.  Adding questions that adequately cover the same subjects would easily increase the size of my guides by 50-75%. In addition, creating realistic questions and answers is extremely time consuming. Adding these to my guides might well double the amount of my time required to create them. Either the price of the guides would need to increase significantly or I would need to take a reduced royalty payment (from the increased page count) while simultaneously putting in more hours to create each guide. I do not believe that most of my readers would consider the practice questions to be worth a significant price hike. I know that putting in that much more time for less return would make me considerably less interested in spending my time creating them.

The second issue is that I am not a proponent of studying for exams by answering practice questions. The goal of preparing for Oracle certification exams is to gain as much knowledge about the tested topics as possible. Focusing on learning the answers to specific questions is a terrible habit to fall into. For example, imagine a chapter in a certification preparation book that has five end-of-chapter questions. On taking the self test, someone misses four out of five. This strongly implies that they did not understand the material and should re-read the whole chapter more carefully.  However, many candidates will simply look up the sections in the chapter that dealt with the four questions they missed. Once they have memorized all five answers, they will be under the impression that they are fully prepared for the real test. In reality, it is unlikely they fully understand much of the information outside what the questions covered. If the actual test questions are much different from the practice ones, they are likely to get them wrong.

Finally, the largest problem that I have is that end-of-chapter exercises cannot be close enough to the real test to provide candidates a reasonable measure of how prepared they are for the actual exam. There are several reasons for this, including the following:

  • Generally people answer the questions right after reading the material. This does not mean they will recall it days later when sitting for the exam.
  • Often (as noted above), the exam candidate will check their information in the chapter before answering.  The real test is not open book.
  • The questions are grouped together by topic (i.e. what the chapter was about). This makes answering them easier. The real exam jumps around topics randomly.
  • End of chapter questions have no time constraint.
  • There is absolutely no pressure involved. Test takers are not worried about failing, or whether they really put down the right letter on that one really hard question they answered a few minutes ago.

In my opinion, the primary benefit that practice tests provide is to help exam candidates determine if they are ready for the real test. This is the 'confidence' spoken of in the original question. For that confidence to have any basis in reality, the environment of the self test must be as close as possible to the real exam. It should have the same number of questions and the same time limit as the real test. Exam candidates should treat the practice just like the real exam and not peek in the book or use the internet to help on questions they do not know. I would even recommend taking it at the same time of day and at a terminal close to what will be in the testing center. If all of that is done, the results of a well-written practice exam may be a reasonable approximation of what candidates will get on the actual test.

What all of this adds up to is that I do not believe that end-of chapter exercises provide sufficient value to candidates to be worth the considerable effort that would go into adding them. This is not to say they have zero value. End-of-chapter questions make more sense for the Oracle Press series. The chapters in those books are considerably longer than those in my guides. It is possible -- especially late at night -- to read a complete chapter without actually absorbing the information. If you miss a majority of the practice questions and use that feedback to recognize you should re-read the chapter (perhaps the next day), then they can add value to your study process.