Thursday, June 30, 2011

PnL: June 2011


Profit for the month, June 2011 is +36.08%

Wednesday, June 29, 2011

wednesday

Sunday, June 26, 2011

sunday

Have you ever played a session where fish is sitting to your left, calls every preflop raise you make and you still manage to win almost every hand at showdown? I just had one of those sessions.

On a different note...

I mentioned in yesterday's post that I managed to replace the cmos battery and cleaned my cpu fan. Last night, the cpu fan started to sound weird, then the pc shutdown by itself. Restarting, the pc did not even boot up, so I opened my box to see what happened.

The heatsink was loose. Somehow yesterday, the brace that holds the heatsink in place seemed to have been damaged. I reattached the heatsink to the cpu and the pc was working again. This time though, the box has to lie on its side or the heatsink will fall off. I may have to learn how to take apart the motherboard and screw in the heatsink brace, but that's a different troubleshooting lesson for me. I am going to stick with this workaround for now.

Saturday, June 25, 2011

CMOS battery and CPU fan

I waited two hours for this acer pc to boot up this morning, and it didn't. I decided to finally replace that CMOS battery. I went to the two dollar store and got a pack of 5 CR2032 batteries for only NZ$2.50.

To make sure I was going to do this right, I watched a few youtube videos on how to replace the batteries and at the same time, clean the CPU fan. One hour after I got the batteries, I am making this post from said acer pc. Everything seems to be in order. The box is a lot more quiet now.

I only used whatever household item I found to clean the inside. I literally had to pick up the dust cakes inside the box by hand. Next time, I am going to get one of those compressed air canisters to clean the inside.

I used to think that this was tough work that needs professional attention (aka, pay someone else to do it). Times are different now. There's google and youtube. I figured that in case I break something, or can't get it to work, then I'll find someone to fix this for me. In the meantime, this is a great opportunity to improve my hacking skills.

The refurbished IBM pc is showing PXE-E61 error on boot-up. That's next on my troubleshooting TODO list.

Now, on to some poker sessions.

Friday, June 24, 2011

friday

It's tough going through a correction. I start to doubt myself, wondering if I am better off getting a job instead. I have second thoughts if I should play like a nit instead of wasting my chips investing in my metagame.

I have been through this before. The best thing for me to do is observe these thoughts and learn something from the situation. Sometimes the fish makes his/her nuts on the river; sometimes I do. Keep the faith.


Atm I am back at 2nl. Soon as I have breakout, I take shots at 5nl.

Friday, June 17, 2011

Thursday, June 16, 2011

thursday

It is getting too much of a drag to log in to windows vista. The only reason it's still in my pc is because of poker. From pushing the on button to getting my poker stuff up and running would take from ten to fifteen minutes. Longer if vista has system updates.

I found FOSS alternatives that I can use in linux, but they are not as reliable as the ones I am using with vista atm. I am looking into the possibility of tweaking them to make it work, but that will take some time. For now, I am keeping vista.

I found a workaround for the acer boot issue. I think the problem with it is that the cmos battery needs replacing. So what I do is I turn on acer and let it just sit there for a few minutes. Somehow, either the battery gets its juices going and acer boots up on the restart. I don't see anything out of the ordinary once everything is up and running.

I feel anxious about opening my pc to replace that battery. I am doing research on how to do it, and there are plenty of resources out there that really helps. Maybe one of these days if/when the workaround I have doesn't work anymore, I am going to have that battery replaced.

I finished the ibm pc linux install a day later. By finished I mean, after running linux, I updated drivers, installed all mine and the kids' choice of applications, sync files, etc. The only thing that isn't in the ibm pc is skype.

With all these technical stuff behind me, it was only today that I started playing poker again.

Tuesday, June 14, 2011

second pc

I got the kids an old pentium 4 pc off trademe.co.nz. It has windows xp, but I removed that and installed linux instead. It's now in the process of updating to v11.04 natty narwhal.

I chanced upon the auction this morning. There was only one bid with less than half hour to go. For NZ$80, I am getting a p4 IBM desktop, 2ghz with plenty of memory and all that. It comes with a keyboard, mouse and speakers. Typically, for that same price, you only get a refurbished desktop without monitor and all that--just the box.

For this particular auction, the seller refurbished the pc and threw in everything else to get it working. I waited for the last minute before the auction closed and put in my bid--and won.

I was able to play one session early this morning. Soon as I finished that, I went off to get the pc. I got back two hours later and since then, I have:
- removed windows xp
- made a full install of ubuntu 8.04
- now updating to ubuntu 11.04

Hopefully the ubuntu update will finish tonight. I will install our favorite FOSS apps tomorrow morning.

The kids are starting to use the pc and internet for their homework more often. I think it's a good investment to get them their own. This way, I get to run my poker business when needed without taking time away from their learning. I ordered a router from the same auction site. It might arrive thursday or friday.

Monday, June 13, 2011

Download & Install VLC Media Player in Ubuntu 11.04 Natty Narwhal via Command Line [PPA]

While I have tor disabled, I decided to update VLC--

1. Open the terminal and add PPA by executing the following command,

sudo add-apt-repository ppa:n-muench/vlc

2. Update and install VLC media player along with few other plugins,

sudo apt-get update

sudo apt-get install vlc mozilla-plugin-vlc vlc-plugin-pulse

3. Done.

source-- Download & Install VLC Media Player in Ubuntu 11.04 Natty Narwhal via Command Line [PPA]

ubuntu - can't connect to server

I found TableCrab off google yesterday. It works thru wine, and has to update mine before installing tableCrab. I probably worked on it for two hours last night and couldn't get it to upload. I tried changing software server from nz to main and back several times--nothing worked. It was way past my bedtime and decided to let it go.

The following morning, I thought to myself that the problem might be connected with TOR. I changed network proxy settings to disable tor, updated my repositories by following the instructions from here.

It worked. My version of wine is now updated to v 1.3.21.

Now I am going to look into getting tableCrab into linux and have a look at its source code. Tablecrab is written in python. I don't get why it has to run inside of wine. I am not sure I want to use it, but I am sure I can learn a lot from reading its code.

...and I am going to reactivate tor.

Friday, June 10, 2011

Payment Processors - Which One is Best?


After fumbling for a few days getting my sea legs back, I think I finally got my game going again.

I became wary of using my credit card to add funds to my poker account. I tried moneybookers, and after several attempts, I gave up and used visa to fund my account. The problem with moneybookers was that, after adding funds to that account, I couldn't find a way to send them to pokerstars.

So I went into the pokerstars platform and clicked on cashier. I then selected moneybookers, entered my email address and amount to deposit. Pokerstars then opened a website from moneybookers asking if--
a. I wanted to use my existing credit card
b. use a different credit card
c. bank transfer
d. fund my account

The moneybookers account was already funded. I used my credit card to put money into the account and moneybookers charged me 1.8% for it. Now it is asking me the options above as to how I wish to make a deposit with pokerstars.

To make the long story short, I ended up withdrawing everything from moneybookers back to my credit card. Now the same thing is happening again--it has been 24 hours and the money is yet to show up in my bank. I have to wait 7 days before moneybookers support will do a follow up.

There must be a better way to do this.

Thursday, June 9, 2011

Programming Poker AI

Programming Poker AI

This article was originally published in the “Inner Product” column in Game Developer Magazine, November 2005

I recently programmed the AI for the World Series of Poker, developed by Left Field Productions and published by Activision. I started out thinking it would be an easy task. But it proved a lot more complex than I initially thought.

This article for the budding poker AI programmer provides a foundation for a simple implementation of No-Limit Texas Holdem Poker AI, covering the basics of hand strength evaluation and betting. By following the recipe set out here, you will quickly become able to implement a reasonably strong poker AI, and have a solid foundation on which to build. I assume you are familiar with the basic terminology of poker.

TEXAS HOLDEM

The goal of any game playing AI is twofold. The primary purpose is to allow the player to have a fun and enjoyable experience. The secondary purpose, subordinate to the first, is to play a strong enough game to provide sufficient challenge to the majority of players in your intended audience.

POKER DATA TYPES

You will need an implementation of the following data types. I'm going to describe them at the bit/byte implementation level, leaving the high level abstraction up to you.

A "suit" is an integer in the range 0..3, where 0=Clubs, 1=Diamonds, 2=Hearts, 3=Spades

A "rank" is an integer in the range 0..12, where 0 = 2 (deuce), 1 = 3, 11 = King, 12 = Ace. This is the cards in a suit arranged in rank order

A "card" is an integer in the range 0..51, hence
card = suit*13 + rank.
Suit = card/13
Rank = card%13

A "Hand" is a 52 bit data type, where each bit represents a single card. This can be stored as four 16 bit words for ease of use, where each 16 bit word represents the potential cards in one suit (using 13 of the 16 bits) (figure 1)

A "Hand Type" is an integer representing the type of poker hand you have, where 0= no pair, 1=pair, 2=two pair, 3=trips, 4=straight, 5=flush, 6=full house, 7=quads, 8=straight flush.

ENCODING HAND VALUES

A "Hand Value" is a 32 bit integer representing the relative value or strength of any hand of cards. By comparing two hand values, you can see which hand is stronger in a game of poker.
The hand value can conveniently be represented as a series of six 4-bit nibbles, where the most significant nibble represents the Hand Type, then the next five nibbles represent the different ranks of the cards in the order of significance to the hand value. (figure. 2)

Example 1: AH QD 4S KH 8C is a "no pair" hand type (sometimes called a "high card", or in this case "Ace high"). So, the hand type nibble is set to 0. The remaining nibbles in the Hand Value are filled out with the ranks of the five cards in descending order. (A, K, Q, 8, 4), which translated into rank indices: 12,11,10,6,2 (or C,B,A,6,2 in hexadecimal), and when combined with the hand type (0) in the high nibble, gives us a 32 bit integer: 0x000CBA62.

The individual suits of the cards are basically ignored in the final hand value. The only time suit is significant is when it contributes to a flush. Also, note the top two nibbles of the Hand Value are always zero.

Example 2: 4D JD 3D 4C AD is a pair of fours, with Ace, Jack, Three kickers. The hand type is a pair, (type 1), then the ranks follow, starting with the rank of the pair, then the ranks of the kickers, so 4,A,J,3, which gives us 0x0012C910.

Example 3: 7C, 6C, 5C, 4C, 3D is a straight (type 4). More specifically it's a seven high straight. The only rank of import here is the seven (rank 5). So the hand value is encoded as 0×00450000. We save ourselves a bunch of instructions in ignoring the four low cards after we've determined it is a straight.

Look at the resultant hand values of the above examples, you can clearly see how the better hands always have a higher hand value, making determining the wining hand a simple comparison.

CALCULATING HAND VALUES

What we now need is a function that takes a hand, and returns a hand value. This involves determining the hand type, then inserting the nibbles for the hand ranks, as above.

A hand is four words (clubs, diamonds, hearts, spades) of 13 bits each. 13 bits can be arranged in just 8192 combination, which means we can accelerate the evaluation of a hand by pre-calculating 8K tables of things like the number of bits set in a (13 bit) word (if you have five or more of the same suit, then you've got a flush), or the highest card of any straight in the hand. You can also pre-calculate a table of the highest five cards from a particular bit combination, which you can then use to set the kicker cards.

If you calculate ranks = (hearts | diamonds | clubs | spades) then the value ranks is a bit-field with a bit set for every card rank that you have at least one of. The number of bits set here is the number of unique ranks you have. We calculate the number of bits in each of hearts, diamonds, clubs and spades, and subtract the number of bits in the unique ranks, giving the number of duplicated ranks, to be used as the basis of determining what type of hand you have.

Example: if you have 2D AS AH 2C 2H, you can very quickly determine that you have five cards, that there are just two unique ranks, and hence you must have either a full house or four of a kind. A few more simple tests will determine exactly what you have. The entire evaluation function will consist of tests like this, gradually whittling down the possible hand types.

Since the function consists mostly of bitwise operations, table lookups and simple comparisons, it is going to be very fast. It's also very amenable to fine tuning optimization, and the exact implementation will depend on the target architecture. You may be able to take advantage of some processor specific instructions to greatly improve the efficiency.

CALCULATING HAND STRENGTH

Hand strength is the probability that you will win the hand, given your hole cards, the community cards, and the opponents who remain in the hand. Hand strength is a floating point number between 0.0 (certain loss) and 1.0 (certain win). For example, a HS of 0.33 means you have a 33% chance of winning.

The easiest and most flexibly way of calculating the HS is to simulate the progress of the game a very large number of time, and count the number of those times you win. Say you simulate the game 1,000 times, and in the simulation, you win 423 games, then you have a high degree of certainty of having an approximate HS of 423/1000, or 0.423.

The procedure for simulating a game is very simple:

Create a pack of cards
Set score = 0
Remove the known cards (your hole cards, and any community cards)
Repeat 1000 times (or more, depending on CPU resources and desired accuracy)
Shuffle the remaining pack
Deal your opponent's hole cards, and the remaining community cards
Evaluate all hands, and see who has the best hands
If you have the best hand then
Add 1/(number of people with the same hand value) to your score (usually 1)
End if
end repeat
Hand Strength = score/number of loops (1000 in this case).

To be more accurate, we have to run our simulation with people dropping out if they are dealt hole cards below a certain threshold. In practice, the determination of if a player stays in or not in a simulation is a probabilistic function of the strength of their hole cards, their table position, their stack size, the blind size and their previous behavior. For now we can just modify the simulation, so after dealing the opponents hole cards, remove any non-blind players with hole cards worse than, say, a pair of sixes. While not particularly elegant, it will still give you a useful number.

POT ODDS

The pot odds number is the ratio of your bet or call to the size of the pot after you bet (the amount you will win). For example, if the bet is $20, and there is $40 in the pot, then the pot odds are 20/(20+40) = 0.333.

RATE OF RETURN

Rate of return is the "on average" proportion of how much you will multiply your bet by, if you stay in the hand.

Rate of Return = Hand Strength / Pot Odds.

The base strategy we implement is to mostly stay in hands with a rate of return greater than 1.

THE FOLD/CALL/RAISE DECISION

For each round of betting the computer needs to decide if it is going to fold, call or raise (The FCR decision). Ignoring the question for the moment of how much to raise for now, then given a Rate of Return (RR), it's possible to provide a very simple (yet useful) mapping between RR and FCR.

If RR < 0.8 then 95% fold, 0 % call, 5% raise (bluff)
If RR < 1.0 then 80%, fold 5% call, 15% raise (bluff)
If RR <1.3 the 0% fold, 60% call, 40% raise
Else (RR >= 1.3) 0% fold, 30% call, 70% raise
If fold and amount to call is zero, then call.

Don't pay too much attention to the precise percentages listed above, the numbers will depend on the way you calculate your hand strength, and you'll want to vary them depending on which betting round you are in. You will also want to vary these numbers to create players with different personalities.

Using this very simple mapping between the RR and the FCR decision can give you a surprisingly reasonable and entertaining player. They will tend to play strong hands, they will occasionally bluff, they won't scare easy if their hand is good, and they will abandon weak hands when raised, and they will stick around on a reasonable chance of a flush or straight draw, making for entertaining gameplay.

The fact that none of the percentages is 100% is also important. That means you can never deduce the hand strength of your AI opponent based on their actions (unless they fold, where the information does not really help you). If they raise, then they could have any kind of hand strength - probably a good one, but it might be the 1 in 20 times when they are bluffing with a very weak hand.

STACK PROTECTION

The simple rules above work well when your stack of chips is large and the blinds are small. However as your stack shrinks and the blinds increase then the amount of money you need to commit to stay in a hand can become a very substantial proportion of your stack. Also, occasionally other players might go "all-in", betting their entire stack of chips, so we need some logic to prevent the AI from making bad calls when short stacked.

Say you have AD, 2D and the flop is QC, KC, 2C. So you have a pair of twos, but there is a possible flush out there. There is $500 in the pot and the bet is $100 to stay in against two player, but it's your last $100. The pot odds are 100/600 = 0.1666, your hand strength is 0.297, so your rate of return is about 1.8. So if you could play this situation over and over again you would make on average an 80% profit each time. However, it's your last $100, and you have about a 70% chance of loosing everything. Don't make that bet!

To handle this we can use a simple heuristic, along the lines of:

"If my proposed bet will substantially commit my stack, then don't do it unless I have a strong chance of winning"

which might be implemented in part by:

"if (stack- bet) < (blind * 4) and (HS < 0.5) then fold"

Meaning if the call would leave you with less than four times the big blind, then don't call unless you have a greater than 50% chance of winning.

Poker is a complex game, with a surprisingly large number of different types of situations like this that you have to handle somehow. I recommend you have as few special cases as possible, as it reduced the risk of an exploit being introduced into the game via some obscure special case. However, you should anticipate a number of heuristics (rules of thumb) being hard coded into the AI logic.

TESTING POKER AI

Playing a quick single table game of Texas Holdem takes around 30 minutes on average with human players. Ideally you would perform your testing by having human players play against the AI and trying to find problems with it. Unfortunately, due to the random hands being dealt, it's very easy for one player to simply get lucky and win the game with sub-par logic, or even flawed logic. I've found it takes at least ten games to begin to get a clear picture of the qualities of an AI player, and more like a hundred games to be really sure. This often creates an unreasonably burden on the testing department, and introduces a very long delay in getting feedback on AI changes.

The solution is automated testing. The AI should be set up so that different variants of AI can play against each other in a very high speed set of games. You should also code a few simplistic poker AI's into the mix, such as an AI that always goes all in, or another that simply always raises with a hand better than a pair of fives. Then you set your AI loose against these opponents, and make sure that it wins the appropriate percentage of games. If you coded your evaluation and simulation appropriately, then you should be able to simulate an entire game in about a second. (You might want to reduce the iterations of the simulation a bit to speed up testing).

The best use of your human testers is to try to get them to find an exploit of the AI, then you can codify this exploit into a temporary AI opponent to include in your test suite. You can then tweak your AI until it defeats the exploit, while still being able to defeat all the other (standard) opponents.

MORE WORK

What I've set out here is just a foundation for poker AI. By following the process laid out here you will get a reasonably strong and entertaining opponent. Here's a quick list of the topics you might want to look into

� Pre-flop hand strength tables
� Opponent modeling.
� Implied Odds.
� Personality modeling
� Positional play
� Probabilistic search space
� Game theory and Nash Equilibrium.

RESOURCES:

- Sklansky, David, The Theory of Poker, 1999, Two Plus Two Publishing. – Provides various discussion of pot odds, implied odds, etc, with many heuristics that might be useful.
- The University of Alberta Computer Poker Research Group:
http://www.cs.ualberta.ca/~games/poker/ A number of research papers on implementing poker AI.
- Hold'em Killer, Evin Peretz, http://www.holdemkiller.blogspot.com/ – A blog on implementing poker AI.
- Poker-Eval, http://freshmeat.net/projects/poker-eval/ - A GPL Licensed poker hand evaluation library.


source-- Cowboy Programming » Programming Poker AI

june 2011 baseline


I am having difficulty adding funds to my account at this time. Visa seems to not work anymore, moneybookers, too complicated; uKash doesn't work in nz atm. I'll just continue playing. If any amount gets added to this account, it'll be discounted as flow at the end of the month.

PS
3 hours later... added $33 to the account.

Wednesday, June 8, 2011

back in business

I found an email from our bank a moment ago. They've found my money and has already been credited to my account-- Y E S !

After more than a month of waiting, everything is back in order. I tried to deposit the funds back into the poker business, but the account seems to have been blocked by the bank for online transactions. I am going to have to call them about this tomorrow morning.

updating

After more than a month of learning python, I decided to check in on vista and get my software updated. Thirty minutes after turning on my pc, I am have yet to finish updating HM.

I decided to ease back into playing poker to get my algorithms right.

I haven't played in a long time that I forgot which button on my gamepad/tableninja effects which action. I am going to have to play one table in the meantime to check if everything works out alright. Soon as things get ironed out, I will add tables one at a time until I have the maximum running.

Monday, June 6, 2011

monday

Google Doodles: 2011 April - June

It is the queen's birthday today here in nz--a public holiday. We spent the day in the mall. Boring. In fact, everything has been kinda gray lately. I miss playing poker. Learning how to code gets dry and dreary. I feel like I am back in school. I am in a way, except that I am homeschooling myself learning python.

I wrote my first code recently. It's a unit conversion tool. User inputs numbers to convert and specifies the conversion unit as output. The app has more than 700 lines of code. The app runs beautifully, and I am adding more units to convert into it when I feel like it. When it's all in there, I am going to work towards pythonifying it and making the code shorter, and more efficient.

I emailed the bank a few days ago with regards to my missing funds. Pokerstars gave me a receipt number that can be used to trace the funds. If the bank replies that the money is not there given the tracking numbers, then I can use that email reply and send it back to pokerstars. It's like a tennis match, except that I am simply a spectator--feeling like I am at the mercy of one or the other.

The lesson for me here is this--next time, don't use visa.

atm, I am looking at bitcoins and the possibility of playing poker on that side of the fence. But there's nothing there. Codingthewheel had a post about these recently that got me looking into the rabbit hole. At this time, nothing makes sense. I can't even get my dollars converted into bitcoins. I'm not motivated. There's not a lot of poker happening on that side of the currency. Betco.in, when I checked in at that site, only had one active table with only two players heads up with one player sitting out. Googling that site, I read somewhere that it was hacked some time ago(I think codingthewheel mentioned that).

There are other poker sites that use betcoins, but they are other ghost towns where nothing happens. I am not closing the door on this, only that it's not going to take as much attention from me as it did these last few days.

Time to get back to learning.