Feedback on Oracle Certification Prep Practice Tests

It has been almost four months since the initial release of the Oracle Certification Prep practice tests. I have been keeping a close eye on the results throughout this period. The test engine being used has some excellent analytical reporting that has been very helpful in locating questions that test takers consistently get wrong (or right). This capability has proved invaluable in helping to ensure that test questions are in the 'Goldilocks' zone by being neither too hard nor too easy.

In addition, I have been tweaking the overall difficulty of the tests with the intent of making them just slightly harder than the corresponding production Oracle exams. Feedback from a number of the people who took a practice test and then went on to take the real exam seems to indicate I am very close to my goal in that respect. Test takers are fairly consistently reporting a production exam score at or slightly above the one they received on the Oracle Certification Prep practice tests.

This afternoon as part of this feedback I received a request that deserved to be covered here:

"In my opinion your practice test was closer in difficulty and overall feel to the Oracle test than Kaplan's. One suggestion: The feature on the Kaplan test allowing you to review your results along with correct answers & explanations was really useful. I would love to know what I missed on yours and why."

I take feedback very seriously and am extremely interested in fulfilling requests where possible. The Oracle Certification Prep website was actually built in response to feedback on the earliest study guides I wrote. Someone indicated in an Amazon review that additional preparation materials were required beyond what was in the guide.  I agreed and created the website to meet that requirement. The Oracle Certification Prep practice tests were created largely because readers complained that the study guides contain no practice questions.

Unfortunately, in this particular case, the fact that the practice tests do not allow test takers to review their results was not an oversight but rather a design specification. There are several reasons why the feedback is given at a topic level rather than a question level. These include:

Gap Analysis -- In an earlier article on this blog: Using Gap Analysis when Preparing for Oracle Certification Exams, I discussed how practice tests fit into that activity.  The following is a quote from that article:

"Practice tests are designed almost entirely to perform gap analysis. Many have a 'study mode' where the answers to questions and a brief explanation are shown.  However, once you use the study mode, the test becomes less useful for identifying gaps. You will have learned to answer that specific question, but not necessarily another on the same topic. Retaking the test will make it seem as if the gap has been closed, however."

Currently there is only one version of each of the exams covered, although eventually there should be two or three versions of the more popular exams. If test takers are shown the answers to missed questions -- that test becomes effectively useless for gap analysis.  I have had a number of people take a test, fail it, then retake it several weeks later and pass. If the exams allowed test takers to view missed questions -- the retake score would potentially be very misleading. I do not want certification candidates to gain false confidence from my practice tests.


Tunnel Vision -- One of my problems with certification tests in general is that exam takers focus entirely too much on specific questions. The questions on a certification exam are intended to act as a measurement of your overall knowledge of a topic.  Knowing the answer to question 'X' does not mean a test taker understands the topic that question represents. Memorizing the answer to a specific question that was missed on a practice test can still leave the test taker open to other questions on the same subject. My professional opinion is that it is more valuable for a test taker to be told that they have a weakness in a general area. My hope is that they will perform a broad review of the information in that area to strengthen their skills rather than perform a laser-focus review of a single question.


Keep it Real -- More than anything else, the Oracle Certification Prep practice tests are intended to be as much like the real exams as possible. There is no question-level feedback on the real exam. The goal (obviously) is to pass the production exam on the first attempt. However, if that does not happen, test takers need to be able to use the feedback provided to determine what they need to review before rescheduling the exam. The fact that the practice tests provide feedback that is equivalent to the production exams is intended to be part of the training.


The upshot is that there is no plan to change the level of feedback provided by the exams. I understand that some people would prefer more information and I understand why. That said, my opinion is that not providing the answers is the better option from a learning standpoint.