Law of Similarity?
#21
Posted 2020-November-28, 18:30
However if you have to divide a suit up between 3 hands, the number of available cards 0 to 13 determines the distribution and possible lengths of those hands
Trivial example 0-4, say, cards left the chance of anyone having length in that suit is 0
Any more cards and the chance of anyone having length in that suit increases
As I said, I caan't be bothered to even think what kind of distribution it is. How you can arrange n cards in the longest remaining suit (say) between 3 etc and how the distribution changes as n increases etc
.... and I gave a trivial example of how it works in the simplest case I could think of with 2 hands, 2 suits and 2 cards each
.... actually I know you could have a 2 hand, 2 suit, 1 card each game that maybe also demonstrates it but I wanted a game with more than card each
But maybe to make it more convincing we need to start with a 4 hand game and the most basic case etc
#22
Posted 2020-November-28, 19:57
I think that it is a mistake to start by examining the precise algorithms that are used to assign cards to hands.
Shuffling is more complicated than it seems (just look at Knuth).
I think that a better way to proceed is to look at shuffling in the abstract. What are the characteristics of a "good" shuffling algorithm?
To me, a good shuffling algorithm is one which
1. Is capable of generating any / all bridge hands
2. The likelihood that any possible bridge hand is generated is uniform
As some of us have noted, with this as a starting point, it is trivial to show that the law of similarity is nonsense.
Equally significant, if you are starting someplace different (say some specific BAD shuffling algorithm) you can probably justify (pretty much) anything that you want.
"I assert that the Law of Similarity holds true because my mother does a dreadful job shuffling hands and I normally play bridge against my mother and her friends and it feels like this happens" really isn't a testable theory.
#23
Posted 2020-November-29, 09:44
It allows you to input a shape (e.g. 5-4-3-1) and see the probability that any given suit (or number of suits) has the same shape.
The following table shows for each common shape first the generic frequency, then the probability that 0,1,2,3,4 suits have the same shape.
All numbers are rounded percentages, the last five come from running the script.
If the Law was true then the last five numbers should be close to 0 100 0 0 0.
[Note that Culbertson fails even to consider that more than 1 suit may have the hand shape and the example he gives would fail if it happened].
Shape %Freq %0 %1 %2 %3 %4
4333 10 34 42 20 2 1
4432 22 20 40 29 10 1
5332 15 31 41 23 4 1
6322 6 48 39 11 8 0
6331 3 62 32 6 0 0
5431 13 34 48 15 3 0
5422 11 37 45 10 2 0
5521 3 67 27 6 0 0
4441 3 70 26 4 0 0
5440 1 69 29 2 0 0
If the numbers are correct then it's easy to see why Culbertson may have 'felt like it happens' with 5431 and 5422, although even there it happens in precisely one suit less than half the time.
#24
Posted 2020-November-29, 13:10
pescetom, on 2020-November-29, 09:44, said:
Just to reiterate, this is not what the law says. Culbertson was well aware that it is not true for mathematically random hands. The law applies to real-life shuffled hands only:
Quote
#25
Posted 2020-November-29, 13:59
smerriman, on 2020-November-29, 13:10, said:
I too imagine he was well aware, and probably had noticed less than 50% too. That did not convince him to mention either fact, nor to mention the issue of shuffling.
#26
Posted 2020-November-29, 14:12
pescetom, on 2020-November-29, 13:59, said:
What are you talking about? Did you read the actual article?
It specifically mentions that the whole law is about shuffling, and not the pure mathematical side - the whole first 5 paragraph intro is about this and nothing but this.
#27
Posted 2020-November-29, 15:14
smerriman, on 2020-November-29, 14:12, said:
It specifically mentions that the whole law is about shuffling, and not the pure mathematical side - the whole first 5 paragraph intro is about this and nothing but this.
No I had not read this 1954 attempt at self justification, thank you. I shall do so.
I did read and cite the actual article from 1933 that stirred up the whole embarassing mess however, and that is what I am talking about.
It makes no reference to dealing or modification of probability, to mention a couple of terms I glimpsed in your article.
#28
Posted 2020-November-29, 16:53
I reckon you could definitely challenge "more likely than not", even use of "likely" in the wording but maybe "marginally more likely" (in the sense of, there exists an epsilon > 0 etc) is more appropriate - thats the forumlation I've been working on - and I have needed to simplify to two suits and two hands or a very simplified gaame to demonstrate the obvious symmetry.
Also, are we talking similarity or identicality in shape etc Obviously with 4 hands and 4 suits identical shape is an option - is it that specific?
Does there need to be the same number of suits as hands? Is the number of cards in each suit irrelevent (eg 4 hand 4 suit 5 card game etc)
I'm sure similar (in the broader sense) is more likely the more extreme (or freaky - Pavlicek) your hand is
But come to think of it, its obviously more likely for identical shapes
I will go even further and claim that the chance that everyone has the same shape hand as yours increases too
- in fact this is the most simple and obvious case
(Restrict it to ordered hands to simplify 1,2,3,4 . There is only one comintaion where all 4 hands have an identical shape to any particular shape etc. (5-4-0-1, 1-5-4-0, 0-1-5-4,4-0-1-5), once you have that shape the probablity everyone else has that shape has gone up - because you have many fewer deals to divide by) But this very simple case is not the interesting one. The more interesting one is whether overall freakiness has gone up in the other hands too but that seems obvious
It reminds me a bit of the first time I heard the weather forecaster discussing the SOI and the forecast for rainfall. The forecast was 50% chance of above median rainfall. I used to think that was strange and obvious until I heard at other times that it was on 30% chance of above median rainfall etc
Oh, and getting back to the obsession with shuffling I maintain the distributions are not affected by shuffling at all
But while I find shuffling theory rather tedious (I dipped into a paper for a few seconds) - one thing that fascinates me about packs of cards developing character over time is whether they can every be restored to being interesting packs after (entropy???) has increased so much they have become boring - I wonder looking at the world at the moment if shuffling theory and entropy and irreversible processes apply to the world at the moment - and that sadly it is irreversible. No chance of it every being fun or interesting again
#29
Posted 2020-November-29, 18:02
pescetom, on 2020-November-29, 15:14, said:
I did read and cite the actual article from 1933 that stirred up the whole embarassing mess however, and that is what I am talking about.
It makes no reference to dealing or modification of probability, to mention a couple of terms I glimpsed in your article.
Ah, I didn't realise there was an earlier version to the one in his book.
You're right that he didn't provide any form of justification in that original excerpt, but it's pretty well agreed that human shuffling leads to flatter deals than 'normal', so I can't say it would be a massive surprise if there were other corollaries like greater 'symmetry'. And of course all hands back then were human shuffled.
#30
Posted 2020-November-29, 20:33
thepossum, on 2020-November-29, 16:53, said:
I reckon you could definitely challenge "more likely than not", even use of "likely" in the wording but maybe "marginally more likely" (in the sense of, there exists an epsilon > 0 etc) is more appropriate - thats the forumlation I've been working on - and I have needed to simplify to two suits and two hands or a very simplified gaame to demonstrate the obvious symmetry.
Also, are we talking similarity or identicality in shape etc Obviously with 4 hands and 4 suits identical shape is an option - is it that specific?
Does there need to be the same number of suits as hands? Is the number of cards in each suit irrelevent (eg 4 hand 4 suit 5 card game etc)
I'm sure similar (in the broader sense) is more likely the more extreme (or freaky - Pavlicek) your hand is
But come to think of it, its obviously more likely for identical shapes
I will go even further and claim that the chance that everyone has the same shape hand as yours increases too
- in fact this is the most simple and obvious case
(Restrict it to ordered hands to simplify 1,2,3,4 . There is only one comintaion where all 4 hands have an identical shape to any particular shape etc. (5-4-0-1, 1-5-4-0, 0-1-5-4,4-0-1-5), once you have that shape the probablity everyone else has that shape has gone up - because you have many fewer deals to divide by) But this very simple case is not the interesting one. The more interesting one is whether overall freakiness has gone up in the other hands too but that seems obvious
It reminds me a bit of the first time I heard the weather forecaster discussing the SOI and the forecast for rainfall. The forecast was 50% chance of above median rainfall. I used to think that was strange and obvious until I heard at other times that it was on 30% chance of above median rainfall etc
Oh, and getting back to the obsession with shuffling I maintain the distributions are not affected by shuffling at all
But while I find shuffling theory rather tedious (I dipped into a paper for a few seconds) - one thing that fascinates me about packs of cards developing character over time is whether they can every be restored to being interesting packs after (entropy???) has increased so much they have become boring - I wonder looking at the world at the moment if shuffling theory and entropy and irreversible processes apply to the world at the moment - and that sadly it is irreversible. No chance of it every being fun or interesting again
Since I have the data, I spent a little while putting together a program that determines these basic statistics for each of the 39 generic hand shapes (4=3=3=3 to 13=0=0=0):
(1) The numbers & proportions of deals that have 1, 2, 3, 4 and no hands of that shape;
and, analysing the sets of deals that contain the shape in question,
(2) The percentages (and numbers) of such deals that have 2 or more hands of that shape, and
(3) The percentages (and numbers) of such deals categorised by the longest suit in one (or more) of the other hands.
Results for the two shapes (7=2=2=2 & 4=3=3=3) you originally cited are:
Total deals: 53,644,737,765,488,792,839,237,440,000 Target shape: 4=3=3=3 18,904,824,864,906,126,262,212,096,000 deals with 1, 2, 3 or 4 of target: 1: 15,538,600,726,161,191,018,436,096,000 = 82.19384 % of all such deals; 2: 3,078,920,993,459,221,886,976,000,000 = 16.28643 % 3: 237,337,380,888,197,990,400,000,000 = 1.25543 % 4: 49,965,764,397,515,366,400,000,000 = 0.26430 % Length of longest suit(s) in the other 3 hands: 4: 1,341,268,488,045,803,116,800,000,000 = 7.09485 % 5: 9,389,152,279,359,996,649,015,910,400 = 49.66538 % 6: 6,567,757,945,719,475,336,421,990,400 = 34.74117 % 7: 1,457,245,678,194,666,886,759,833,600 = 7.70833 % 8: 143,135,495,072,392,782,498,739,200 = 0.75714 % 9: 6,173,444,779,468,438,792,320,000 = 0.03266 % 10: 91,533,734,323,051,923,302,400 = 0.00048 % --------------------------------------------------------------------------------------------------------------------------------- Target shape: 7=2=2=2 1,091,600,331,330,190,676,219,596,800 deals with 1, 2, 3 or 4 of target: 1: 1,082,537,511,172,874,920,302,796,800 = 99.16977 % of all such deals; 2: 9,049,166,455,516,001,717,760,000 = 0.82898 % 3: 0 = 0.00000 % 4: 13,653,701,799,754,199,040,000 = 0.00125 % Length of longest suit(s) in the other 3 hands: 4: 34,329,156,878,471,495,040,000,000 = 3.14485 % 5: 465,402,880,409,496,995,338,813,440 = 42.63492 % 6: 444,973,635,287,073,217,557,872,640 = 40.76342 % 7: 126,372,546,662,990,136,932,966,400 = 11.57681 % 8: 19,117,164,834,880,139,265,638,400 = 1.75130 % 9: 1,360,547,526,432,774,670,848,000 = 0.12464 % 10: 43,938,720,545,275,887,851,520 = 0.00403 % 11: 461,010,300,641,525,606,400 = 0.00004 %
For example:
(1) of all the 1.9x10^28 deals with a 4=3=3=3 hand only 16.28643% have 2 such; for 7=2=2=2 the proportion is 0.82898%.
(2) of all the deals with a 4=3=3=3 hand the longest suit in one of the other hands is 7 cards in 7.70833% of such deals; for 7=2=2=2 the proportion is marginally higher at 11.57681%. Whilst there are differences, if you plot the bar graph of the percentages for each length the patterns of the distribution for each shape are markedly similar.
The data does not support any sort of "symmetry" law. Of course, this is what one would expect: one is concerned with the ways in which the 39 cards not in the "target" hand are distributed between the other 3 hands: in the 4=3=3=3 case those are 9 cards of one suit and 10 of each of the others; for 7=2=2=2 it's 6 of one and 11 of each of the others. Just as 5 cards in two hands tend to split 3-2, so n cards over 3 hands tend to distribute relatively evenly; it's the same effect. It's not surprising that finding 7+ in one of those hands is relatively uncommon.
Edit: Tables put into 'code' format.
#31
Posted 2020-November-29, 22:14
PeterAlan, on 2020-November-29, 20:33, said:
The data does not support any sort of "symmetry" law.
Given the number of hands involved and the probabilities we are talking about its unlikely any small simulation would show anything up. Also when O mentioned symmetry, the principle will only obviously show up with a trivial simple example but I am sure (without having studied that speciality) that it is there
I was hoping it could be discussed/analysd and/or understoood through a more analytical mathematical formulation
#32
Posted 2020-November-30, 01:51
thepossum, on 2020-November-29, 22:14, said:
I was hoping it could be discussed/analysd and/or understoood through a more analytical mathematical formulation
I don't understand exactly what you're saying, but the numbers I gave above are from the complete 53,644,737,765,488,792,839,237,440,000 deal space and its detailed sub-structure, and not any sort of simulation. In that sense, it's exactly the sort of thing you're calling for.
#33
Posted 2020-November-30, 04:00
PeterAlan, on 2020-November-30, 01:51, said:
EDIT. I will return to your post in the morning. Too much whisky. The following is still being edited
What I thought we were talking about Peter is the change in probability of either an identical or a distributional hand given any particular shape as compared to the average chance etc
And just out of curiosity did your code actually enumerate all those hands
I will accept a mathematical argument if there is anyone reading this site with that level of maths
For starters what we are talking about is a change in probability of the order 10^-28 to 10^-some other reasonable integer
And we are also discussing a level of symmetry in such a complex way that you wouldnt know if those results were symmetric or not
But while I think of it I was pondering that the law of similarity is getting close to a tautology. It applies with every hand and as soon as you pick up your hand and look at it (strictly when it is dealt) the probability of similarity has gone up. Can you show me the change in that probability with your program
And it doesnt come down to an estimate, an approximation, a confidence interval. There is actually a number
#34
Posted 2020-November-30, 06:05
thepossum, on 2020-November-30, 04:00, said:
For starters what we are talking about is a change in probability of the order 10^-28 to 10^-some other reasonable integer
And we are also discussing a level of symmetry in such a complex way that you wouldnt know if those results were symmetric or not
I don't think that anyone is going to do a bunch of random work in the hopes that you are happy with it
You need to make a specific claim.
For example, lets assume that you get dealt the following hand
♠ KQ87
♥ JT87
♦ K95
♣ QJ
It's certainly possible to enumerate all the ways that the remaining 39 cards might be divided across the remaining three hands (and the likelihood that the cards will be divided in such and such a way).
You seem to be asking whether the hands will deviate from the expected likelihoods.
Please explain what you think will be more likely (and ideally why this might come to pass)
#35
Posted 2020-November-30, 08:42
Seriously what is all your obsession with trying to prove me wrong. None of you have demonstrated that capability at all
So keep copying and pasting my stuff as much as you like to try and undermine me. Keep trying to undermine my arguments and my knowledge in every issue because I challenge this site's BS ona regular basis. I'm not wasting more time here. Go and pick on some innocent victims with your flawed analysis and argument
#36
Posted 2020-November-30, 08:45
thepossum, on 2020-November-30, 08:42, said:
Seriously what is all your obsession with trying to prove me wrong. None of you have demonstrated that capability at all
Ha ha ha ha...
#37
Posted 2020-November-30, 08:51
I'm out of here. I have enough real problems in my life and difficult people without having to worry about saying something careless in a discussion about a silly Bridge problem
I apologise to anyone who ever took any offense at any of my hot-headed (but not meant in any real way) comments in any forum any where in the world
Stop making people's lives difficult please.
#39
Posted 2020-December-01, 08:26
smerriman, on 2020-November-29, 18:02, said:
You're right that he didn't provide any form of justification in that original excerpt, but it's pretty well agreed that human shuffling leads to flatter deals than 'normal', so I can't say it would be a massive surprise if there were other corollaries like greater 'symmetry'. And of course all hands back then were human shuffled.
I just read that 1954 "Contract Bridge Complete" excerpt, thanks. Culbertson at age 63 is still a fascinating showman, and astute to bring in the human shuffling smoke screen so that simple mathematics can no longer expose the weakness of his law.
His account of how absent or ineffective shuffling can quickly lead to a "cold deck" is inspired. As you say, this phenomenon is recognised and well agreed to produce flatter deals than 'normal'. It is not obvious why this should coincide with an increased probability of a coincidence ('extraordinary', as he says) between hand pattern and suit pattern between hands, however, and even less so why this increase should extend to those unbalanced hands that do get through the net.
He then states "A balanced hand-pattern is generally accompanied by a similar balanced pattern in at least one of the four suits". Simulations with (truly) balanced hands suggest that this is more likely than not, quite often in more than one suit, which is hardly a great basis for important decisions. But it looks like his distributions are behaving normally here, human shuffling or not. Below he adds "It is also extremely likely that the balanced type is formed by the remainder of your own long suit" and later "The more freakish the hand-pattern, the greater the expectancy of a similar freakish distribution of the longest suit". In other words, your long suit is the most probable suit in which you will find your hand shape reflected. This is of course no surprise at all if the deal is randomic - your longest suit has less cards to distribute among the other hands - and is born out in my simulation with randomic deals. Again it looks like his distributions are behaving normally, despite the voodoo of human shuffling.
The problem here of course is that as your long suit gets longer, not only the chances of symmetry in a side suit tend rapidly towards zero but the chance of symmetry in the long suit itself drops rapidly too (reductio ad adsurdum - consider the case of 10-1-1-1, where the probability of symmetry in the long suit is 22% and in a side suit is 0.12%). With random deals this is already evident with a 6 card suit and glaringly obvious beyond that. Apparently there are no such problems with manually dealt hands, as Culbertson's examples of application of the "law" often involve very unbalanced shapes. Here he first discusses 6511 (<14% chance of symmetry in simulation) or perhaps 5521 (around 28% in simulation) - the text and example do not match. Then he says "With a hand-pattern 7-4-1-1 I am not so happy about my seven-card suit, for it is astonishing how often it will break 7-4-1-1." . Astonishing indeed if it often does so (around 15% in simulation). Suddenly he is in a different universe as far as probability is concerned, the magic of human shuffling.
You're also right of course that we cannot disprove this "Law" in the context of manual shuffling. That doesn't mean that we have to give it the benefit of the doubt, however.
#40
Posted 2020-December-08, 04:02
Today there's no excuse since you can just google/simulate/wolframalpha the correct probabilities, but even Culbertson didn't have a good excuse, he could just have read Borel's book.