Tuesday
Feb072012
The Final Version - first look
Tuesday, February 7, 2012 at 13:09
Here are some never-before-seen photos of the first and last pages of an actual FV list, or to be more exact the only actual FV list in existence.
The loose-leaf format is not essential. Any sort of notebook, paper or electronic will do just fine.
Reader Comments (122)
Interesting. How do you explain the great difference in the margin by which the two methods win?
It's all academic though, because the way I use Colley's rule in the Final Version doesn't allow for it to be replaced by the Rule of Three or Four or whatever.
Hopefully I'll have some time tomorrow to look into the Rule of Three/Four.
(a few minutes later)
Okay, it's late, so maybe I'm confused, but ... why is the Rule of Three faster than Colley's (as claimed by FSE)? Buying a house by Rule of Three, I have to find three houses that meet my spec, never two, always three. Colley's will often terminate at the second qualified house, no? I don't think it's obvious which is faster; you'd have to look at various cases of how the houses were sequenced and average over them. Rule of Four would be even slower.
Then again, it is late.
The first thing I noticed is that my original analysis of Colley's Rule was actually an analysis of "Rule of Two." That's right, take FSE's Rule of Three, cut it down to a Rule of Two, and do the math. You will get exactly what I got for Colley's. Which means, my Colley's is not Colley's!
I think that when I glossed over the items/houses encountered between the benchmark and the final selection, I ended up ignoring combinatorics that affect the result. That is, my linear staircase histogram underestimates the number of ways to reach higher-valued outcomes, because it does not account for the various combinations of the losing houses that crop up after the benchmark. At the time, I thought they didn't matter, but now I think they are going to act as a weighting factor that pushes our probability curve much further toward the higher values. It will make Colley's much more effective than I initially reported.
For example, for N=3, we have the following six possible sequences:
123 --> 2 (Colley's result)
132 --> 3
213 --> 3
231 --> 3
312 --> 3
321 --> 3
And so, P(3) is 5/6, not the 2/3 predicted by my original analysis.
For N=4,
2 ways to get 2:
1234, 1243 --> 2
5 ways to get 3:
1324, 1342, 2134, 2314, 2341 --> 3
17 ways to get 4:
1423, 1432, 2143, 2413, 2431, 3124, 3142, 3214, 3241, 3412, 3421, 4123, 4132, 4213, 4231, 4312, 4321 --> 4
(Double-check: Total "ways" = 2+5+17=24, which is 4-factorial, representing the entire outcome space.)
So the correct P(4) is 17/24, whereas my staircase histogram calls it 3/6.
Correct P(3) is 5/24, vs. my 2/6; correct P(2) is 2/24 vs. my 1/6.
Yeah, I really underestimated Colley! Imagine how much more enamored I'll be once I get it right!!
So, I will start over, no shortcuts, using the entire N-factorial outcome space. I can't say how it will compare to Rule of Three, but it sure looks like Colley's combinatorics are going to pile up very steeply. Since you two are all over the number crunching already, I'll stick to the equations. But first I'll go to sleep. Really.
I would not look at the first item, and then seek a more suitable candidate. I would search for the first "good enough" item, and then seek a more suitable candidate. The corresponding Rule of 3 would be rather more difficult: "Find 3 suitable candidates and choose the best". Definitely not just a pick 3 task any more.
I'm short on time today (ironically, I've procrastinated my real work!), but once I have time I will repeat the simulated comparisons of this Colley variant to the Rules of 3 and 4.
<<Should I start a new blog for this discussion?>>
I am in favor. Then I won't feel bad for spewing equations at people who don't care about math or those who already feel perfectly comfortable with Colley's Rule.
When I started this, I was skeptical about the quality of Colley's result and wanted some sort of validation to buy into using it when FV comes out. By now, I have seen enough validation and it is becoming a hobby. I will continue posting if anyone wants to, or I will gladly keep it to myself. Either way, I'm going to finish the equations (which are coming along nicely), even if someone settles it by digging up the textbook answer! ;)
I have to agree with Alan B, Colley will not result in mental rebellion (at least in me) compared to the Rule Of Three, or Four or N. (I’m going to call it/them, the Rule Of N – or RON for short). You see, I work as a program manager, managing anywhere between 15 and 20 IT projects, totaling between $15 and $20 Mil at any given time. I’m currently also pursuing my Master’s degree in Information Systems (IS), and an Acquisition course provided by my employer. I also have a family that relies on me as a father and husband, I write Science Fiction and Fantasy, and finally – I have ADD. In short, I have a ton to juggle, and I can’t rely on my brain alone to juggle it for me. I have used several systems, including Steven Covey’s 7 Habits, GTD, and virtually every system Mark has offered. I have used the Rules of Three and Four, as well as Colley’s for at least a month each.
Now I’m no mathematician, but I do know systems. I define an IS as “people using technology, or a tool, to orchestrate and manage a business process.” IMHO – AF, SF, DIT, DWM, FV and their ilk all fit this definition of an IS on a personal level. Now given that people drive IS, not machines, I will not rely on mathematical proofs and statistical analyses to choose between Colley and RON. My criteria have to do more with how a person would use each rule for the purpose of selecting the next task. I have several such criteria, but many do not form a basis for comparison.
For example, using the AF method of processing and doing what “feels ready” does not work for me. Maybe it’s my ADD, or maybe I’m just human, but using the “feels ready” method for a period of several months resulted in my doing tons of little things and ignoring the big things. My list became an aid to, and a written testament of my procrastination. I need a method that foils my procrastination, rather than one that allows it to continue unfettered. Therefore, my method of processing must not base itself on what I ‘feel’ like doing more, but rather on what I ‘value’ more. Both Colley and RON do this and so this criterion of value based judgments does not form a real basis of comparison.
However, I do have two criteria that form a decent basis for comparison: strain on “mental RAM,” and the speed of list traversal.
Studies show that the average person can consciously store an average of up to 7 pieces of information in their working memory (“mental RAM”) at one time. All scientific notions aside – what I need in terms of strain on mental RAM boils down to this: The less mental work needed to accomplish the process of selecting the next task, the better. RON states “look only at the first N possibilities, choose the best one.” So my mental processor must accomplish the following:
1) Look at the first task – determine if it is a “possibility” (ie: “Can I do this now?”)
2) Continue iteratively checking until I find the first possible task
3) Continue iteratively checking from there until I find the second possible task
4) …And then the third, and then the fourth, or the Nth – These N possible tasks may at this point, be found on multiple separate pages
5) Since all are possibilities, I must now hold all N tasks in my mind, and perform a value based evaluation to determine the best one
For example, considering a Rule Of Four; I would have to evaluate several tasks for the possibility of doing them now – enough tasks to identify 4 possibilities. I must then hold these four in my working memory (that can generally hold a max of 7) and use the remaining 3 “slots” of memory to perform value based judgments between them. I know it’s a simple matter to calculate how many comparisons that would be, but like I said, I’m not a math guy so I don’t care enough to do it. Anyway, after all that work, I would discard 75% of the possibilities that I expended effort finding, just to select one task to work.
Colley on the other hand would break out like this:
1) Look at the first task – determine if it is a “possibility” (ie: “Can I do this now?”)
2) Continue iteratively checking until I find the first possible task, and name it the benchmark – it will be discarded later
3) Continue iteratively checking from there until I find the second possible task
4) Holding the benchmark, and the next possibility in my working memory, use the remaining 5 “slots” to decide which has greater value
5) Either stop here, or discard the possible task and repeat steps 3 and 4
RON requires me to do enough work to identify N possible tasks every single time, guaranteeing that 66% (for Rule Of Three) or more (for Rule Of Four or more) of that work will be discarded every single time I select a task. Additionally, it requires that I compare N tasks together at the same time, or carry a (potentially changing) running favorite through a search for the remaining possibilities.
Colley guarantees that I will discard only those tasks necessary to identify the next best thing. Additionally, it requires that I carry only one (unchanging) benchmark task in my working memory through each comparison, and that I compare only 2 tasks at a single time. Colley 1, RON 0.
Now when I mention list traversal speed, I refer to the amount of the list read (whether acted upon or not). I care about list traversal speed because tons of work flies unevaluated onto my list. It comes from many directions, and I don’t have the time to evaluate everything upon entry (hence GTD didn’t work out too well for me). If my list traversal speed < new unevaluated work arrival, then I end up “chasing the end of the list.” My focus centers on certain parts of my list, and I don’t get much of an occasion to visit the other parts of my list. Therefore I begin to lose touch with the full scope of my work, and without that context, I have more trouble saying “no” to tasks I just don’t have the bandwidth to handle at the time, and my list just fills and fills. I spend more time adding to my list than working it off.
With RON, I must engage in the above mental process (which takes time) to discard at least 66% of that effort upon choosing a task. Upon finishing that task, I cross it off, and then I must begin the next task selection process. This requires that I either recall what possibility #N was from the previous task selection process (that I likely discarded and forgot about as I worked on the task I selected) so that I can begin to identify the next N tasks to compare, or I can simply begin at the next possibility from the last action’d task. However, doing so would carry the likelihood that the next possibility from the last action’d task was a possibility from the previous selection process, meaning that I not only considered possibilities that I discarded in the previous selection, but also that I may re-consider them in the next selection process.
For example: I consider possible tasks A, B, C, and D. I choose to do task B. I then cross B off. I could then try to remember which task was task D, so I can begin locating tasks E, F, G, and H. I could also just begin identifying possibilities from B onward, identifying tasks C, D, E, and F for comparison. In the first case I have to keep track of what D is or remember it somehow. In the second case I would be re-evaluating tasks C and D. Either way slows down my list traversal speed.
Colley has no such additional processing necessary, nor does it risk re-evaluating a given task in two selections in a row. Also, there’s a chance that the benchmark chosen may have a very high value, and so I will have to traverse a larger portion of my list just to find the next better task. So Colley has less processing to hamper progress, and can occasionally result in a higher degree of traversal in a single given selection. Therefore, using Colley results in faster list traversal overall. Colley 2, RON 0.
In my personal experience, RON results in more mental strain than payoff, and slow list traversal. My experience using Colley indicates that it results in less mental strain for a decent payoff, and quick list traversal. My vote: Colley’s all the way.
Mark, I am a big fan of you and your work. Of all the task management systems I’ve tried, none have handled my task variety, task load, and scatter-shot focus as elegantly as your systems. And of your systems, what works best for me is DWM2 (on a 4-day “Letter” cycle, and 15 day “Number” cycle rather than the traditional 7 and 30) processed via Colley’s Rule. Of course DWM2 is really DWM1 in disguise, which of course is really DIT in disguise… The curse of ADD involves never having enough focus to see anything (not even trivial things) through to completion – thereby dooming the poor sufferer to a life of always trying, and always failing. Using your systems as my tools I have managed all of my responsibilities well for years, succeeding in things trivial and grand, without resorting to the use of prescription medication at all. This feels to me like a monumental triumph, the lifting of a heavy karmic curse – like shaking the fruit loose from the tree of possibilities, or finally surfacing from the deep dark depths of the despair of failure, for a breath of fresh air. Your work means a lot to me personally, and I shall wait on the edge of my seat to snatch your next book hot off the press.
-Miracle
The Final Version as I have it at the moment will, I think, fit all the requirements you mention in your post. It's certainly doing a lot for *me* at the moment. As I've mentioned before I am not using Colley's rule in a conventional way.
I've used the RON with N as 2, 3, 6 and 10 in the past, and you're right in that it's nothing like as powerful as Colley. It's not just a matter of finding a good choice of task in the abstract, but also of psychological readiness. And the FV majors on that.
So it's not just giving priority to the important tasks that tend to stay last in SF, and a better choice of what to work on ;)
A time management system is not just a way of identifying the "best" task to do next. That might work if we were computers, but we're not. The system has also to ensure that the task it identifies is one that we are psychologically ready to do.
I'm really convinced I've got it now. My original "cascaded lottery" concept was right on target, but I went astray when I drew its histogram as a staircase. Now, fixed.
The results are astounding! Colley's performs quite respectably versus the tedious Secretary's Rule and very similarly to the lightweight but un-Forsterish Rule of Three. I'll start off with an example, because it makes a very clear pattern.
N=5:
P(1) = 0
P(2) = 1/5 * ( 1/4 ) = 1/20
P(3) = 1/5 * ( 1/4 + 1/3 ) = 7/60
P(4) = 1/5 * ( 1/4 + 1/3 + 1/2 ) = 13/60
P(5) = 1/5 * ( 1/4 + 1/3 + 1/2 + 2 ) = 37/60
(add 'em up: 60/60)
For arbitrary N:
P(n) = 1/N * ( 1/(N-1) + 1/(N-2) + 1/(N-3) + ... + 1/(N-n) )
with special cases P(1) = 0 and P(N) = P(N-1) + 2/N
Plugging in N=3 and N=4, I get the same results I posted earlier when I wrote those cases out in full and counted them up by brute force. There was not a single equation to go wrong that time, so I'm taking it as confirmation of my new formula.
To generalize beyond specific values of N, I made a spreadsheet of N=100:
The luckiest 10% of cases land above the 98th percentile of desirable items.
The luckiest 37% land above the 88th percentile (compare to Secretary's Rule, in which they get their top applicant).
The luckiest 50% land above the 81st percentile.
The unluckiest 15% land at or below the 50th percentile.
The unluckiest 10% land at or below the 42nd percentile.
The unluckiest 5% land at or below the 30th percentile.
(The point of N=100 is *not* that N needs to be large. I just wanted a fine-grain result that does not depend on a specific N. These figures give the right general idea for small N too.)
What the variables mean:
N = # of items (e.g. houses) that meet your specification. Call them "qualified items."
P(n) = probability of the n-ranked item (out of all N) being selected by Colley's Rule,
n=1 refers to your least desirable of the N qualified items,
n=N refers to your most desirable of the N qualified items.
To use Colley's in real life and obtain these results, you do not need to know the value of N or whether it is large or small.
The logic behind this formula:
We are drawing the benchmark (we'll call its rank "b") from a 1-of-N lottery, followed by drawing the winner from a 1-of-(N-b) lottery in which items not beating the benchmark have been removed. The first lottery simulates wandering at random until we encounter the first qualified item, and the second simulates the fact that whenever we first beat the benchmark, any of the higher-ranked items are equally likely to do it. There is a special case when we draw the very best item first and are unable to beat it. The stuff I said earlier about "full N-factorial combinatorics" etc. was technically right but overly complex and not ultimately helpful.
Mathematically, the winning approach was this:
P(n selected, given benchmark b) = 1/(N-b), for b<n. It is zero for n<b or n=b<N, and it is 1 for n=b=N. Then we obtain P(n) by summing over all values of b for the given n.
Comparison to Rule of Three (R3):
R3 luckiest 6% land above 98th percentile.
R3 luckiest 32% land above 88th percentile.
R3 luckiest 50% land above 79th percentile.
R3 unluckiest 12.5% land at or below 50th percentile.
R3 unluckiest 5% land at or below 37th percentile.
(I computed using P(failing to exceed p percentile) = p*p*p)
--> Rule of Three performs quite similarly to Colley's Rule. Colley's is better at picking top choices, while R3 is better at avoiding bottom ones. Earlier comments about R3's usefulness for human to-do lists and psychological readiness are well taken and of course not reflected in these numbers.
Colley's Rule Outcome Statistics
Example for N=100:
75.745 average value
21.95 standard deviation
6.18 average # qualified items examined
(though 2/3 of the time, you will examine no more than 3 items)
Keep in mind, this is not a normal distribution (bell curve), so most conventional wisdom from stats class does not apply. The average is not the most likely result, as the most likely is N itself, and you cannot use the std. dev. to talk about three-sigma-this and four-sigma-that. These are computed means, nothing more.
Despite the 6-ish avg. # of items examined: half the time, you will examine only 2 items, and two-thirds of the time, you will examine no more than 3 items (compare to Rule of Three/Four/N). You reach the sixth item only 1/5 of the time. And strangely, that is all true regardless of N, thanks to the peculiar Harmonic Series (below).
These item counts (and the value N) do not include the unqualified items that you would reject in a real-life trial (house shopping) for failing to meet your minimum specification. But if you are comparing to other methods (Rule of Three, etc.), those same unqualified items would have to be passed over as well.
For general N:
Average value
roughly 3/4 of N (think of the middle value that beats the middle value)
actually 3N/4 + 3/4 - 1/(2N)
Standard deviation
roughly N * (7N/6 - 1) / 24 ... for large N ... as if that helps ...
actually 7N*N/144 - N/24 - 49/144 + 7/(12N) + 1/(4N*N)
Average # qualified items examined
actually 2 + 1/2 + 1/3 + 1/4 + ... + 1/(N-1), for N>2
roughly, umm, well ... http://en.wikipedia.org/wiki/Harmonic_number
intuitively, the chances of not beating benchmark in k items are 1/k, in which the highest of those k is discovered first (note how this does not depend at all on N).
Fine Print
- This is all for a pool of N "qualified" items, meaning that they meet your specification: they are all at least barely acceptable. Unqualified items have been ignored completely.
- An item's "value" is its rank, 1 to N, according to your preference scale. In practice, you do not know an item's numerical rank when you encounter it, but the rank exists for purposes of analysis, so long as you are able to identify your favorite of any two items.
- To estimate how many unqualified items would clutter up the process, you would have to posit a density of qualified items among the entire pool. For our task lists, there is no generally meaningful way of doing that, but the same items would have to be passed over using any other algorithm vs. Colley's.
Dots by several items - I do that. I avoid multitasking but when interrupted (I answer my phone by Caller ID of a trusted few who depend on my timely advice), and I start something else, it goes in the list with a dot too. When I return to the list I look at the dotted items first to quickly get back to where I was. I don't like to leave tasks undone out of distraction or forgetfulness, only by intention.
"Funding" items - Two identical and adjacent. I've never done that either. Similar but different "Funding" item appears later - When that happens, I recognize I've got a project going even if I didn't realize it when I jotted the first item.
Don't answer anything here Mark, I like the suspense. I just had to remark on the tea idea; that's a Wholehearted Yes if I ever saw one. I'll think about that all day!
P( top x ) = x - x*ln(x)
where x is from 0.0 to 1.0, indicating a percentile range.
Now, anyone can explore the Colley's landscape in a few seconds with only a calculator! (I can hear you all cheering ... or is that the dishwasher?)
For example,
P( top 12% ) = 0.12 - 0.12*ln(0.12) = 37% chance,
meaning that we have a 37% chance of landing in the top 12 percentiles.
Results are nearly identical to my N=100 figures posted earlier.
At the lower end of the scale,
P( bottom x ) = x + (1-x)*ln(1-x)
For a middle range, it is simplest to measure from the top:
P( top x, but below the top y ) = (x-y) - x*ln(x) + y*ln(y)
e.g., for the second-highest quintile,
P( top 0.4, but below the top 0.2 )
= (0.4-0.2) - 0.4*ln(0.4) + 0.2*ln(0.2) = 0.2446
= 24% chance of landing in the second-highest quintile
This time, the math comes from a cascade of continuous random variables:
Let benchmark "b" be a random real number from 0 to 1 (uniformly distributed).
Then, Colley's selection is another random real number from b to 1.
And presto, N has disappeared!
There is one catch: without N, there is no scenario of giving up and settling for the benchmark, which means we have implicitly assumed the search goes on and on until we beat it, no matter what. Logically, this is equivalent to assuming infinitely large N. There's always a tradeoff!
To justify this approach rigorously, we have to imagine a rating scale calibrated to the actual range of existing items (houses). That is, a 1.0 refers to your favorite of the actual existing houses that meet your specification, and a zero refers to your least-favorite of the same. In practice, you would not be able to rate the houses on this scale, since you would not be aware of the range of all houses in advance. But in principle, such a rating scale exists, and that is all we need for analysis.
This is an important point, because otherwise we might imagine that 1.0 corresponds to absolute perfection (it is math after all), and then the "top x" probabilities would be quite misleading. E.g., just because P( top 1% ) is 6%, this does not give you a 6% chance of finding an almost-perfect Mediterranean villa when shopping in the Bronx of New York.
Josef: "Make tea" is fine, but I wouldn't include "Drink tea" lest the tea get cold before you reach that item.
<<Bernie: how does one take practical use of your calculations?>>
Suppose you are thinking, "I'm not sure about this Colley's Rule. I mean, pick the first one and then pick the first better one? Come *on*!! Why, it sounds barely better than a blind lottery, and I'm not letting some *lottery* influence my priorities ..." Then Mark's Final Version comes out, and you can't bring yourself to trust whichever part of it uses Colley's Rule. And then your entire 2012 is ruined.
Well, fear not! Now, with a basic calculator (as long as it has natural log), you can answer your own questions about Colley's performance. That is precisely why I started in on the equations—to convince myself—and then after being quickly converted, I continued on because it was downright fun, and I *know* I'm not the only geek on this list. I am of course joking about 2012's being ruined.
For example, if you have a several-page list with some 100 items (thus "N is large"), you might ask yourself, what are Colley's chances of pulling something in the top half of my preference/priority/whatever-Mark-is-using scale? Use the P( top x ) expression from my last post:
P( top 0.50 ) = 0.5 - 0.5 * ln(0.5) = 84.7% chance of hitting the top half of my preference scale.
What are my chances of landing poorly, in the lowest 10%?
P( bottom 0.10 ) = 0.1 + (0.9) * ln(0.9) = 0.52% (as in, half a percent!)
So it is nothing at all like "barely better than a blind lottery." Rather, it is an impressive amount of leverage gained from a minimal mental effort. So impressive, I can hardly stop toying with it and am at least as excited to see how Mark has managed to apply it as I am to see the Final Version itself. If only I were presently shopping for a house!
If you happen to have nothing like 100 items, you can still resort to my earlier expressions containing "N" to see how Colley's plays out. You'll have to add up chains of fractions, which might get old, or you might get a spreadsheet to do it. The result is very much the same as large N, letting you know that you can completely relax about the whole thing even if you haven't much idea what your "N" is.
Of course, if you have but limited faith in equations posted by strangers on the Internet, you can always go to http://www.random.org/ and see for yourself (thanks to Mark for that link!). Pull a dozen or so random numbers at a time, take the first as your benchmark, and scan for the next better one. It's amazing how high the results come out!
Now, if none of this ever bothered you in the first place—if at most you thought, "Colley's Rule, eh? Okay, I'm game"—then none of my work here will be of the slightest use to you. Except to pat you on the back for having been right all along. In particular, you do not need and cannot use any of this stuff when actually applying Colley's Rule or (I can't imagine!) the Final Version. These formulas are only for analysis.
Now if there were only some way to apply this to trading the markets...
<< Now if there were only some way to apply this to trading the markets...>>
There is! It just doesn't tend to make any money. ;)
Glad you enjoyed the equations.