10 scenarios, graded as you go, with the exact grading principle for every answer. Start free, no sign-up.
Timed mock assessment
30 scenarios in 36 minutes, drawn from every topic. No answers shown until the end, then a full score breakdown, your weakest topics and a review of what you missed.
Prices are in US dollars, plus tax where it applies. Stripe, our payment processor and the seller of record, adds tax at checkout.
All sales are final. If something is really wrong, such as a duplicate charge or a purchase that does not unlock what you bought, write to support@annotationprep.com and we will put it right.
One-time payment. Nothing renews.
You have an assessment ticket for this practice test. Start the mock assessment to use it.
An LLM writes a JavaScript function to sort numbers:
```js function sortNums(arr) { return arr.sort(); } ```
What happens when you call `sortNums([10, 2, 1])`?
Returns `[1, 2, 10]`.
Returns `[1, 10, 2]` because `Array.prototype.sort()` converts elements to strings and sorts lexicographically by default, AND mutates `arr` in-place.
Throws a `TypeError` because numbers need a comparator.
Returns `[10, 2, 1]` unchanged.
Show the answer
B. Returns `[1, 10, 2]` because `Array.prototype.sort()` converts elements to strings and sorts lexicographically by default, AND mutates `arr` in-place.
Without a comparator `(a, b) => a - b`, JS `.sort()` sorts by UTF-16 code units (`'10' < '2'`). It also mutates the input array in-place; use `[...arr].sort((a, b) => a - b)`.
Why must you flag this code in a coding RLHF evaluation?
`list.append()` returns `None` instead of the list.
Mutable default argument bug: the same `tags` list is shared across all calls that omit `tags`.
Python does not allow two parameters with similar names.
`tags` must be a tuple to use `.append()`.
Show the answer
B. Mutable default argument bug: the same `tags` list is shared across all calls that omit `tags`.
Default parameter values in Python are evaluated once at function definition time. Using `tags=[]` causes state to leak between independent calls. The golden fix uses `tags=None` and initializes `if tags is None: tags = []`.
Principle: Python Best Practices: Mutable Default Arguments
An LLM writes a SQL query to find all users who have NEVER placed an order:
```sql SELECT u.id, u.email FROM users u LEFT JOIN orders o ON u.id = o.user_id WHERE o.status = 'paid' AND o.id IS NULL; ```
Why does this return 0 rows?
`LEFT JOIN` does not allow `IS NULL`.
Putting `o.status = 'paid'` in the `WHERE` clause turns the `LEFT JOIN` into an inner join—any unmatched row has `o.status = NULL`, which fails `o.status = 'paid'`.
`WHERE` must come before `FROM`.
`o.id` can never be `NULL`.
Show the answer
B. Putting `o.status = 'paid'` in the `WHERE` clause turns the `LEFT JOIN` into an inner join—any unmatched row has `o.status = NULL`, which fails `o.status = 'paid'`.
Filter conditions on the right-hand table of a `LEFT JOIN` must go in the `ON` clause (`AND o.status = 'paid'`), not in `WHERE`, otherwise unmatched rows (where `o.id IS NULL`) are filtered out.
Principle: SQL Joins: LEFT JOIN Filtering in WHERE vs ON
An LLM writes an async loop:
```js async function fetchAll(ids) { const out = []; ids.forEach(async (id) => { const data = await fetchItem(id); out.push(data); }); return out; } ```
What does `await fetchAll([1, 2, 3])` return?
An array of 3 fetched items in order.
`[]` (an empty array) because `Array.prototype.forEach` does not await async callbacks.
A `Promise` that rejects.
`[undefined, undefined, undefined]`.
Show the answer
B. `[]` (an empty array) because `Array.prototype.forEach` does not await async callbacks.
`forEach` ignores the Promises returned by the async callback and finishes synchronously, so `return out` runs before any `fetchItem` resolves. Use `Promise.all(ids.map(fetchItem))` or a `for...of` loop.
Principle: JavaScript Async: forEach Does Not Await
An LLM attempts to remove all even numbers from a list in-place:
```python nums = [2, 4, 5, 6, 8, 9] for n in nums: if n % 2 == 0: nums.remove(n) print(nums) ```
What does this print, and what is the bug?
Prints `[5, 9]`; no bug.
Prints `[4, 5, 8, 9]`; mutating a list while iterating over it skips elements as indices shift.
Raises `RuntimeError: list changed size during iteration`.
Prints `[2, 4, 6, 8]`; `%` checks for odd numbers.
Show the answer
B. Prints `[4, 5, 8, 9]`; mutating a list while iterating over it skips elements as indices shift.
Unlike dictionaries/sets, mutating a Python `list` during a `for` loop does not raise `RuntimeError`—instead, removing `2` shifts `4` to index 0, so the iterator advances to index 1 (`5`) and skips `4` (and later skips `8`). Use a list comprehension: `[n for n in nums if n % 2 != 0]`.