Tuesday
Feb072012
The Final Version - first look

Here are some never-before-seen photos of the first and last pages of an actual FV list, or to be more exact the only actual FV list in existence.
The loose-leaf format is not essential. Any sort of notebook, paper or electronic will do just fine.
Reader Comments (122)
I am just wildly speculating here, but one way to do this could be to have a rule where you do one task per page. If there are no tasks on the page that can be worked on you dismiss a task from the page. So you either complete a task or you dismiss a task from each page. And you keep cycling through the pages.
In AF1, you'd start working on this item immediately, for as long as you want. When you decide to stop, you'd re-enter the item at the end of the list, and then cross off the dotted item.
For FV, I'm guessing the dot still means "it stands out", but then I have some other ideas of how the item gets processed.
First idea: Mark scans the whole list, putting a dot next to any item that stands out. BUT, no action is taken immediately - the scanning just continues. After a pass through the list, you've got a few items with dots, and having circled through everything, you now come back to the first item. It becomes your Colley's Rule baseline. Skip it, and move on to the other dotted items. When you come to a dotted item that stands out MORE than the baseline, take action, working as long as you want. When done, cross off the item, re-entering at the end if recurring or unfinished. Move forward to the next dotted item, which is now your new baseline. Action the next dotted item that stands out MORE than the baseline. .... If you go through all the dotted items and nothing stands out more than the baseline, that would indicate you don't have enough dotted items, so cycle through your whole list again and create a new set of dotted items. Or something along these lines.
(2) (TOTALLY DIFFERENT GUESS) Read through the items in the list in order. When an item stands out, put a dot next to it, and then work on that item as long as you want. When you decide to stop working: (a) If it's a recurring item, cross off and re-enter. (b) If it's FINISHED, then cross it off. (c) If the task is UNFINISHED, then leave it there, dotted. Then continue cycling through the list. After awhile, you'll have dots on all the tasks you've started but didn't finish. This tells you how many plates you are trying to keep spinning before they all come crashing down. And it also helps unfinished work to stand out more since they already have dots. It also slows down the growth of the list since there's less re-entry.
(3) Some combination of (1) and (2)?
I think the evidence is strongest for guess #2. I don't see any non-recurring tasks that are rewritten. The rewritten tasks are either typical recurring tasks (FTSE, Blog GED, Email) or what appear to be project tasks (Fund Raising Action, which I take to mean "scan the fund raising project and take an appropriate next action").
The only exception I noticed was "Sunset Boulevard", in quotes, which I would guess is the 1950 film. This one is re-entered once or twice, but it doesn't makes sense as a recurring task.
So, I would guess this means that "Sunset Boulevard" is actually a television show that I couldn't find on Google, so it's actually a recurring task.
Or, it was a task that Mark started (i.e., watched a bit of the film). But subsequently, it just didn't stand out anymore at all, and at some point Mark decided to delete it. But sometime after that, he changed his mind and entered it anew.
(For those who don't know, Mark does really apply the "little and often" technique to his film-watching!)
I think the main evidence against my #1 guess is that Mark's solutions are generally much simpler and more elegant than that. Guess #2 has more of the feel of something Mark may have actually invented (or at least tried) - at least in my estimation. :-)
In any case, I find those photos excruciatingly difficult to read, so all my guesses may be hogwash. :-)
In my mind, and given how Mark's past systems work, I would say this has to be assumed as a fact.
I'm also thinking that the sub-list (of stand-out items) might be processed through Colley's Rule according to your idea #1. Even if that's not the case, it sounds on the surface like quite an intriguing system anyway - might give it a try :)
<< I'm fascinated as a newbie to SF, but also exhausted from reading all the nuances, best practices, and rules, and speculation and possibilities, while every 2 minutes there's a new system to learn and adopt. I had high hopes, but I have to ask why the guru is changing systems every few months?>>
There is only one system recommended at present on this website and that is SuperFocus http://www.markforster.net/blog/2011/2/10/rules-for-superfocus.html . That has been the case for quite some time now. All the other stuff is simply discussion and experimentation.
<< It's such a time sink for me to try to follow the master when the master is constantly in flux. >>
Are you asking me to stop developing new and better systems? You are under no obligation whatsoever to follow every suggested new idea on this site.
<< I have tried every system in the galaxy and had thought I was landing with the ultimate, but it's not even stable for one full year. >>
If you buy a computer program don't you expect the developers to keep on improving and developing it?
<< i'm now really depressed at having spent so much time reading about SF and all the nuances and how it's better than everything before and and and and ... I may have to say goodbye and go back to the old ways... >>
Why? The discussions don't affect how well SuperFocus works. Nor do they make your old ways any better than they used to be.
It's fascinating how many different conclusions can be drawn from the same bits of evidence!
<< I admire your ability to create hype >>
and keep you reading this website apparently.
My guess is that too. it's entirely consistent with Mark's comments that it is not about importance or urgence "as such".
In terms of continuing to tease us all, I wonder if Mark is planning to create a pre-release video of further notebook shots against a background soundtrack of "The Final Countdown".. perhaps with the lyrics adjusted to "the Final Version.."
In any case if I keep bating my breath for much longer, I might keel over :)
Very much looking forward to it Mark.
Better keep that oxygen tank handy...don't forget that this will only be FV 1.
And we'll still have to work our way through FV1/AF1, FV1/AF4, FV1/DWM, FV1/SF3 etc... until FV2 comes out, or will that be the Final Final Version, FFV? :)
<< don't forget that this will only be FV 1 >>
No, it's called the Final Version because that's what it is. MY final version. Once it's published that's what I'm sticking to. And that's why it's so important to me to get it right before it's published.
Thanks for the support, but I have to say that it's not come over to me as unpleasant mockery at all.
Not sure what sarcasm you are talking about. My comment was only a bit of fun hinting at the fact that even if it's Mark's final version, the folks on the forum are still going to tweak and tweak.
That's just the nature of these time management systems because often the "problem" is not really time management but self-management (in just using the system consistently as it is).
I still like the pre-release video idea - but then I am in marketing, and I get enthusiastic about such things..
1. Avrum's "hype" comment above.
2. James' comment from Mark's "What to look out for" post.
Even if I were taking a shot at Mark (I was not. I was admiring how Mark was handling the PR of his latest system) how is this doing a disservice to you?
And while Frank was joking, I think there's more than an element of truth in his post. I admire Mark's innovation and improvements (bodes well for my low tolerance for boredom and procrastination (read: changing systems to avoid work), but think consistency is better for (my) piece of mind (GTD, 7 Habits, etc).
<< but think consistency is better for (my) piece of mind >>
and mine too!
Observe the dot pattern of the unactioned items on the last page:
d..dddd.... (stop here. I guess all the following are subsequently added items. d stands for a dot, and the dot stands for...well... no dot)
And look closely at the three following striked items. Only the two first seem to have gotten a dot. The third one seems to have a single stroke, without a dot. Which would give :
d..dddd....dd.
I am certainly seeing what I want to see here. But if I'm guessing right, the dotting process is this one. It's done for each page:
1. Put one left-hand finger on the first unactioned item and one right-hand finger on the last unactioned item of the page.
2. Compare these two items. Determine the one that stands out most (or whatever. I'm no good when it comes to details)
3. With your teeth, pick up your pen and manage to draw a dot in front of the chosen item.
4. Move your left finger down one unactioned item and your right finger up one unactioned item.
5. If your fingers did not cross each other, go to 2. Else, you're done with the page.
I don't remember the exact wording, but Mark said something like the page size is of no importance to the speed at which items get processed. You may think that it indicates that the list is processed as a whole, not page per page. Wrong. The page size has no importance. With this technique, 50% of a page will be done at every pass. Whatever the page size, you will always process half of the items on it, even if they are hard. If you make pages smaller, you will only go from page to page faster.
And about Colley's rule... It says:
1. pick the first one that meets your specification as a benchmark
2. take the next one that's better than the benchmark
1. is unapplicable with Mark's philosophy. If it's on your list, it meets your specification. It is important. It needs doing. ("If it's not important, why is it on your list in the first place?") Therefore, as any of the two is a suitable benchmark, both are valid to be the underdog with regards to the other. It boils down to : Get two. Choose. Period.
Finding the "best decision" is a well-known math problem, sometimes known as the Secretary Problem. Although Colley uses the very first opportunity as a benchmark, the best solution is actually to evaluate and reject the first 37% possibilities, and then take the next possibility that is better than all of the preceding 37%.
If the number of possibilities is small (houses, checking accounts) then Colley might provide similar results, but it's nice to know that you can confidently scale up to any sort of decision.
If you're wondering where 37% came from, it's actually 1/e. For more details, see:
http://en.wikipedia.org/wiki/Secretary_problem
Fascinating. I think the alternative name "Sultan's Dowry Problem" sounds much more interesting than "The Secretary Problem"!
However I get the impression that it's directed at a different problem from Colley's rule in that Colley's rule is a method of making a relatively quick decision with a good chance of a high quality result. The aim is basically to save time and effort.
The Sultan's Dowry method on the other hand is not concerned in the least with saving time or effort but with how you get the highest chance of getting the best result when there is no going back on the individual decisions.
But not necessary in task-picking where you will ultimately pick many and none are truly urgent.
Laurent: "3. With your teeth, pick up your pen and manage to draw a dot in front of the chosen item."
Could that be Mark's key new idea? 8-$
"1" does apply because items can be out of context or not suitable for the time available. If I have 30 minutes between classes, my system needs to be able to select a suitable task. Previous systems couldn't always do this; for instance, with AF1 if the page I'm on does not have a suitable task for the time available, I'd either have to dismiss a page unnecessarily or apply the escape rule.
Using Colley's rule, I can choose as my "benchmark" the "next item that could be done now" (given my location, time available, etc.), and then pick the next item that would be better to do npw.
Actually, I'm pretty sure Colley's rule was also intended for decisions that cannot be revisited.
Otherwise, the decision procedure is trivial. If you look at half of the possibilities and choose the most favorable from them, then you will have a 50% chance of making the optimal choice (but take only half as long as looking through all the possibilities). Of course, you can replace "half" with "three-quarters", "a third", etc. depending on how much you value your time against your need for perfection. There is no right answer, because it's a matter of personal preference.
<<Finding the "best decision" is a well-known math problem, sometimes known as the Secretary Problem. Although Colley uses the very first opportunity as a benchmark, the best solution is actually to evaluate and reject the first 37% possibilities, and then take the next possibility that is better than all of the preceding 37% ... http://en.wikipedia.org/wiki/Secretary_problem>>
Thanks for that article! It also says that your chances of finding the *very best* candidate this way are 37%, which sounds quite impressive. Imagine your chances of hitting the top two or top three ...
As Mark summarized nicely (vs. Colley's Rule),
<<The Sultan's Dowry [a.k.a. Secretary] method on the other hand is not concerned in the least with saving time or effort but with how you get the highest chance of getting the best result when there is no going back on the individual decisions.>>
This made me wonder how much Colley's Rule really trades off, quantitatively, in pursuit of a quick answer, and so I discovered that Colley's gives you that same 37% chance of landing above the 80th percentile (of desirability).
I think this makes a very strong statement for Colley's: by relaxing the benchmarking process from 37% of the entire field to just ... (wait for it) ... one item (!!), the effect is to spread our luckiest 37% outcome over the top 80th percentile, versus the single best item. Definitely a tradeoff, but not bad for a quick pick, eh?
Here is the math (Colley's):
P(above 80th percentile) = 1 - 0.8*0.8 = 0.36
... so 37% cuts off at the 79-point-something percentile.
Cricket,
This Secretary/Sultan's problem addresses your point about job interviews that cannot be revisited once rejected. Unfortunately, their "37% chance of best candidate" relies on large-number analysis, but ...
Applying their 1/e rule to your scenario of 5 interviews, you would take and reject the first two interviews (2 being 37% of 5), then look for something better among the final three. I don't know, I think I'd prefer Colley's again. Throwing away two of five interviews makes me nervous! Throwing away even one of five? Maybe.
<<Here is the math (Colley's):
P(above 80th percentile) = 1 - 0.8*0.8 = 0.36
... so 37% cuts off at the 79-point-something percentile..>>
That's not really the math for Colley's rule. ... You are giving the probability of landing in the 80th percentile if you plan to make your final decision after seeing only two options. No benchmarking is involved here.
I should also point out that Colley's rule, like the 37% rule, was also intended for decisions that could not be revisited.
<< I should also point out that Colley's rule, like the 37% rule, was also intended for decisions that could not be revisited. >>
I'd be interested to know your authority for that statement. Whether or not that is true, Colley's rule works best if you can revisit the decisions - which will be the case in most real-life situations. If no better choice than the benchmark appears by the time you have exhausted all the possibilities then you take the benchmark. The larger the number of candidates the less likely this is to be necessary.
I have used Colley's rule in many situations and it has proved itself in use many times. I cannot imagine how I could use The Secretary method in real life. Just look at the difference in procedure:
In The Secretary you don't even get your benchmark until you have looked at over a third of the candidates. And to do that you have to know how many candidates there are - which will not be the case in many real life situations.
In Colley's rule you don't need to know how many candidates there are, and there is a high probability you will arrive at a solution very quickly - which is the entire purpose of the rule.
>> Using Colley's rule, I can choose as my "benchmark" the "next item that could be done now" (given my location, time available, etc.), and then pick the next item that would be better to do npw. <<
Conceivably, FV could just be as simple as that and that would also accommodate fast-changing conditions such as time available and context.
I remember Mark hinting somewhere that we might think that FV was *too* simple :)
If FV is as simple as "find a task that stands out that you can do now, then actually do the next more-standing-out task you can do now" then what do the dots represent?
Then again, Mark did state earlier: "Colley's Rule does come into it, but not in a way that bears much resemblance to the article you reference" which makes me think this is not the whole story
<<If FV is as simple as "find a task that stands out that you can do now, then actually do the next more-standing-out task you can do now" then what do the dots represent?>>
My best guess is that a benchmark is dotted so that when you reach it twice as a benchmark, you'll know that this is your second time through the list without having an opportunity to do that item. This in combination with some rule could prevent urgent items from being benchmarked twice without a fair shot at doing them.
<<Then again, Mark did state earlier: "Colley's Rule does come into it, but not in a way that bears much resemblance to the article you reference" which makes me think this is not the whole story>>
I was thinking that, too. But I like what we've come up with. It allows the list to adapt to the context and time available, and is accurate at choosing good tasks to do now. I remember reading that you're supposed to walk away from the Final Version feeling like you've used the time as best as you could have, no matter how much time you have. We seem to be at least headed in the right direction.
I haven't spoken to Colley (aka Collee) himself, so my only sources are secondhand discussions. Probably the most authoritative one appears in the Financial Times website:
"The rule assumes you cannot go back to any of the choices you reject" (http://www.ft.com/intl/cms/s/0/3a6fbc42-4fd2-11d9-86b3-00000e2511c8.html#axzz1mTAxdrzh)
I think the reason that you need this constraint is that otherwise the solution is underdetermined. In other words: suppose you could look over as many choices as you wanted, and return to any of them at any time. How many should you look at? Well... that depends. If you want a 50% chance of finding the best choice, then you should look at 50% of the total. If you only have time to look at ten out of N possibilities, then you will only have a 10/n chance of finding the best choice.
When you are not allowed to revisit choices, every option you consider has a large cost - you must discard the previous option - for an uncertain reward. That's why knowing when to stop requires guidance from Colley (or the Sultan). It's a bit like poker, you have play the odds.
But with revisitation permitted, it's more like research than poker. The only cost is your time and the only reward is new information. Those are predictable quantities, and the relationship between them is a simple one-to-one tradeoff. There is never a need to commit to an all-or-nothing gamble. Instead, the decision is straightforward: how much do you value your time against your need for perfection. That doesn't require a mathematical strategy, it just requires you to balance your personal values. In this setting, Colley is no better than flipping a coin every so often and ending your search when it lands heads.
<< If FV is as simple as "find a task that stands out that you can do now, then actually do the next more-standing-out task you can do now" then what do the dots represent? >>
This question led me to think of another possibility, similar to my earlier guesses:
(1) Cycle through the list, looking for an item to stand out. That item becomes your baseline. Do not mark it.
(2) Continue cycling. When you find an item that stands out "more" than the baseline, put a dot next to that item, and work on that dotted item as long as you want.
(3) When you're done working:
(3a) If the task is recurring, cross it off and re-enter at the end of the list.
(3b) If the task is completed, cross it off.
(3c) If the task is unfinished, leave it there on the list, with the dot.
(4) Continue cycling through the list - i.e., start over at (1)
Alternate: Replace "stands out" with "is urgent".
Unsolved: I am still pondering the mechanism by which "importance" and "urgency" get sorted out. I am guessing Mark has some kind of mechanism that sorts this out implicitly, simply by working the system, without requiring pre-processing or conscious decision-making.
But how does it work? It doesn't seem to have any kind of page-based dismissal like AF1 (since it's one long list). There aren't any dates or other indicators anywhere.
Maybe the process of following Colley's Rule works so well that it fine-tunes one's sense of what really belongs on the list, and what doesn't. So after a few passes, you get a strong sense of what is "stale" or "not part of my commitments". And you just delete it.
But I would think there is more to it than that.
<< (4) Continue cycling through the list - i.e., start over at (1) >>
Since the task you've just worked on has now "decreased" in the amount that it stands out by (because you just worked on it), I wonder what would happen if you simply choose that task as the new baseline and go further down the list from that point?
<<Maybe the process of following Colley's Rule works so well that it fine-tunes one's sense of what really belongs on the list, and what doesn't. So after a few passes, you get a strong sense of what is "stale" or "not part of my commitments". And you just delete it.>>
I remember a comment from Mark where I understood that FV will be more like DIT.
So “One day’s outgoing work much on average equal one day’s incoming work”
http://www.markforster.net/blog/2010/1/26/do-it-tomorrow-revision-on-its-way.html
The purpose of Colley's rule is to save time. It gives you a simple method of getting a good result in as little time as possible. And you don't need to know how many candidates are available for it to work.
I don't see how anything you've said affects that.
<<That's not really the math for Colley's rule. ... You are giving the probability of landing in the 80th percentile if you plan to make your final decision after seeing only two options. No benchmarking is involved here.>>
Not at all, FSE. I describe the analysis in detail on page 1 of these replies. I cannot find a way to link directly to my entry from here, but you can find it by clicking back to page 1 and scrolling about 2/3 of the way down. There are a few followups shortly after that shed some more light with a more concrete explanation. Please have a look and let me know if you find errors. We'd better take that to the forum, though, rather than cluttering up this blog post with more equations. I will gladly join you on a new forum thread to iron it out.
Maybe you are looking at the expression 1-x*x and being reminded of the math that arises from a straightforward lottery? E.g., 1-0.8*0.8 does happen to be our probability of winning a 1-of-5 lottery if we are given two chances at it, but that is nothing like the reasoning used in my analysis. The resemblance is pure coincidence.
Instead, I have essentially asked the following:
If 100,000 people, completely independently, made selections by Colley's Rule, what would the luckiest 37,000 people's results look like? The answer is that they would all obtain results in their top quintile of desirable options. I chose 37% in order to compare to the Secretary's case, in which we've learned that the luckiest 37,000 people would actually obtain their very best possible option.
<<... Colley is no better than flipping a coin every so often and ending your search when it lands heads.>>
Colley's is *vastly* better than this! Hopefully a look at the analysis and other comments on page 1 will clear that up. Since Colley's is very effective for small N, you can also confirm this through experiment: write numbers on index cards, shuffle them into a stack, benchmark the top number, and proceed down the stack until you beat it. Make a histogram of your results, and they should beat the pants off of the occasional coin flip or a straight lottery, or even a best of two lotteries ... but I don't recommend 100,000 trials!
Ok, I just ran some simulations in MATLAB for this problem. Yes, I used 100,000 trials per simulation!
Rather than implementing a coin-tossing scheme, I used a very minimalist alternative to Colley's Rule. Let's call it "The Rule of Three":
"Look only at the first three possibilities. Choose the best one among those three."
It's painfully simple, but believe it or not "The Rule of Three" proved superior to Colley's Rule regardless of the total number of possibilities. I can post a detailed table of results if you want, but here is the executive summary:
1) When there is a small number of possibilities (i.e. between 5 and 20), "The Rule of Three" provides much better results on average than Colley's Rule and takes slightly less time on average.
2) When there is a large number of possibilities (i.e. between 20 and 10,000), "The Rule of Three" provides slightly better results than Colley's rule, and takes much less time.
I have to say that the results somewhat surprised me, but there they are. This tends to reaffirm my suspicion that Colley's Rule was meant to be used in a different setting.
I haven't done 100,000 trials, but I tried using the random number generator at http://www.random.org/ to see what happened. I ran 5 trials which the generator set at 1-100 and 5 trials at 1-10. Duplicate numbers were removed.
In only one out of the 10 trials did the Rule of Three produce a better result than Colley's. That was with the 1-10 range setting. In five cases both methods produced the same result. And four results were better using Colley's rule.
As far as time goes, the rule of three was obviously faster most of the time (it would be difficult to see how it couldn't be). Even so Colley's rule produced the same result faster on two occasions.
Maybe I was just lucky.
1-100:
33 37 9 (Both 37)
100 + another 99 numbers (Both 100)
60 34 49 20 42 7 76 (R3 60; Colley 76)
66 53 14 87 (R3 66; Colley 87)
80 16 7 16 78 55 89 (R3 80; Colley 89)
(Colley 3, R3 0, Draw 2)
1-10:
3 4 10 (R3 10; Colley 4)
2 1 5 (Both 5)
6 9 3 (Both 9)
10 + another 9 numbers (Both 10)
7 3 6 5 4 9 (R3 7; Colley 9)
(Colley 1, R3 1, Draw 3)
There seems to be some discrepancy between my results and yours. So far on the various settings which I've used Colley has produced 8 better results to the Rule of 3's two.
There are only six possible situations in the Colley v. R3 stand-off:
Each of the first three numbers must be either H=Highest; M=Middle; or L=Lowest
The possible combinations are:
L M H = R3 wins
M L H = Draw
M H L = Draw
L H M = Draw
H M L = Colley wins
H L M = Colley wins
Therefore:
1) The chances of both methods producing the same result are 50%.
2) Colley is twice as likely as R3 to produce a better result.
However there is a complicating factor which is what happens if the first number is the maximum. In this case some of Colley's wins will become draws. The likelihood of this happening is greater the lower the total number of numbers. With very a low number it would impact severely on Colley's chances of producing a better result than R3 (though only because both would produce the best available result).
My experimental results with the random generator seem to be in conformity with this.
So my conclusions are:
1) If you know there are only a small number of candidates then it doesn't matter much what method you use since you can probably afford the time to look at all of them anyway.
2) If you know there are a more than a small number of candidates, use Colley's rule.
3) If you don't know how many candidates there are, use Colley's rule.
4) If you can't go back or it would be inconvenient, use Colley's rule.
5) Precision and accuracy aren't the same thing.
This seems counterintuitive, but there is a reason for it: when Colley wins, it tends to win by a much smaller margin than when Rule of Three wins. That means Rule of Three can keep a high average score even with a lower win rate.
I think it's pretty indisputable that Rule of Three is faster than Colley. So I immediately wondered whether I could optimize the rule by trading some of its speed for a better win rate, without affecting its simplicity or hurting the average score.
Without further ado, I present to you ... the Rule of Four!
Just like it sounds, in this rule you look at the first four choices and choose the best one. How does it perform?
1) For n > 10, the Rule of Four will provide higher average scores, has a higher win rate, and is faster than Colley.
2) For n < 10, the Rule of Four will provide much higher average scores and a much higher win rate. However, at this range Colley's is slightly faster, though it is still slower than Rule of Three.
I also looked at a new parameter: how often the results are in the top or bottom quintile.
For Colley's, you can expect to be in the top quintile 50-60% of the time, depending on n. For the Rule of Four, you can expect to be in the top quintile 60-80% of the time, depending on n.
Finally, Colley's ends up in the bottom quintile 2-3% of the time. The Rule of Four ends up there <0.2% of the time.
Perhaps I should apply for a research grant in order to study the Rule of Five ...
<< However, in the long run the average score produced by Rule of Three is higher. >>
Is it? In my three experiments, totalling 20 trials, The average results were as follows:
Experiment 1 (1-100)
Colley: 77.8
R3: 68.6
Experiment 2 (1-10):
Colley: 7.4
R3: 6.8
Experiment 3 (1-20)
Colley 16.9
R3: 17.0
So R3 just managed to scrape a win in Experiment 3, but I'd hardly call it a convincing one.
Mean score from 7 trials, choosing from 1-100
Run #1: Colley = 61.0, R3 = 70.9
Run #2: Colley = 95.1, R3 = 85.6
Run #3: Colley = 79.0, R3 = 88.6
Run #4: Colley = 75.9, R3 = 82.9
Run #5: Colley = 76.0, R3 = 78.7
But when I dialed back up to 100,000 trials, the pattern was clear:
Mean score from 100,000 trials, choosing from 1-100
Run #1: Colley = 75.3, R3 = 75.8
Run #2: Colley = 75.3, R3 = 75.7
Run #3: Colley = 75.2, R3 = 75.6
Run #4: Colley = 75.3, R3 = 75.8
Run #5: Colley = 75.2, R3 = 75.8