Hi Tysen,
First, your three evaluation post in rgb have not exactly spurred a rush of discussion of what you are comparaing. If I had to guess, I would think the lack of discussion about your methods is due to the fact that few people can determine what the heck you are evaluating, despite the fact the you have written what amounts to a 20 pages single space describing your evaluation method and said you looked at 2.8 million hands.
I have downloaded your data which contains 13094 hands. The data looks accurate as far as it goes, but we were not privy to the data until you posted the link here (or we simply missed it). Just as you find fault with ZARs approach (and to mine which is not based upon such exhaustive analysis), I think I have found, for me, the flaw in your analysis. You simply count points and say that should be the level.
Zar has convinced the world that a cue-bidding or blackwood is very useful to make sure you are not off two quick tricks for small slam or one quick trick for grand slam. This seems so obvious, that of course I apply this metric to any slam/grand slam evaluation process. And, this is important, ZAR points is very aggressive so such checks are more important playing Zar points than most (all?) other systems. You dont do this in your analysis, so let me show you how this will skew your results, at least as far as a pragmatic approach is concerned using your data. To keep it simple for me, I took the the 54 NS hands that ZAR points would say bid grand slam, but that went down. Of these 54, 29 of were off a cashable ACE. Clearly, I would never consider this a failure, because I believe in using blackwood/cue-bidding. Of the 14 hands that have ZAR count for seven, but cant make but 5 tricks, three were off two quick tricks, and even small slam would not be bid.
It is also interesting to note the type of hand where ZAR count is very high, no quick loser exist, but slam cant make. They are hands like the following (four of the five NS hands with ZAR count 67 or higher that are not off even one quick trick, but which cant make slam. Note the characteristic horrible fit. On the first one, West has 6 spades, east a spade void, and East 6 hearts, West a singleton.
On the second one, East six clubs, west a void. West 5-5 in majors,
East 2-2.
Board three East a void in spades, west five, East 8 clubs, West void. West five diamonds, East singleton.
Board 4, West void in diamonds, East has 8 diamonds, East void in hearts, west five. East singleton club, west six clubs.
Similar misfits exist on a lot of the other hands that Zar points (and probably other evaluation systems) overbids. I think any reasonable computer characterization system should apply the reasonability test of not bidding grand slams off a cashable ACE or two regardless of the point total. I am willing to live with the overbidding on misfits if you must, but even there, I think ZAR fit (and non-fit subtractions) are taken. For instance when playing ZAR points, you subtract 3 points for every trump you are short in partners suit (see Eric's and Zar's discussion). In theory, a void could easily lead to subtracting 9 points. So while you call hand 4 above worth 73 ZAR points, how many points does it have (counting fit points). In diamonds, west loses 9 point, dropping from 73 to 62. It is interesting to speculate if EASTs hand should be devaluated for lack of fit also, he would be minus 9 points for heart fit or 6 points for
♣ fit. I am not exactly sure how such secondary suit subtractions should (would?) work. But if so, that would drop the count even further, to 56, suggesting a final contract of 5, not seven.
Similar minus fit (and plus fit) adjustments need to be made on all hands, what ZAR call his FIT-plus points. I am not terribly surprised that going on sheer ZAR points without a reality check for cashable quick tricks, much less degree of fit, leads to overbidding at slam/grand slam level. This is why, for instance, in my review of the Cavendish thread, I stated up front that reality check would be in effect (not off two aces, not off two quick tricks in any one suit).
But thanks for posting these hands, they will give me some to evaluate using the plus/minus adjustments (and since you have the normal zar points there, I can start from that). To bad you couldn't post more hands, it is fun looking at them, and the analysis of how many tricks can be made in each denomination.
Ben
Help
