ApocBot v0.0

ApocBot v0.0 — The First (not completely working) Open Source MTGO Trade Bot written in C#/.NET!

Now that the fanfare is over, let’s get down to the nitty-gritty.

What was with the delay?

The truth of the matter is that life just has a way of happening. My desire to play Magic The Gathering: Online waxes and wanes through the years and I started this project shortly after moving across the country. So you can say it was the product of my being bored in the evenings Over the course of a few months my interest in this project started to fade as I settled into my new job and had to travel a bit.

I had debated in my mind on how to handle the bot. I know over time it would be a maintenance nightmare as people would require support and updates, which is honestly why I decided to go Open Source with it. For some reason people feel that paying for software implies some form of support! In the end, I just stopped working on it because (a) MTGO is just a hobby, so I wasn’t looking to make money off the bot and (b) I didn’t want to have to constantly maintain and update the code to work with new versions/UI changes as they come out. That’s just the lazy bastard I am

That being said, I still feel this is an important project because it gives the community at large the ability to finally be more open about automation on MTGO (and possibly other games).

So what shape is the code currently in?

Out of the box, the code isn’t going to work. I’ll go into the details below on why/what needs to be updated to get it in running shape.

Other than updating some of the code to work with any UI/screen changes, it should still work pretty well! I know it worked the last time I tried to use it (six months ago?).. so I’m sure with some TLC, someone can get it up and running again.

And it’s free?!

Yep! Anyone is free to download and use the code for ApocBot in their own Magic Online Trading Bots as long as they adhere to the BSD License terms. If anyone is feeling giving, links back to my blog and some Magic Online gratitude would be appreciated! (My MTGO user name is “WndrBr3d“)

Download:

ApocBot and the source code are released under the BSD License. Please review the included “license.txt” file for terms and conditions. For more information on the BSD Licenses, click here.

ApocBot v0.0 (Source)

Requirements: Visual Studio 2010 and .NET 4.0

And now.. the very rough technical details on how it works…

How does it even work?!

I followed the same basic idea as a lot of the AutoIT bots that are out there in the sense that I’m snapping screen shots and determining what’s on the screen by creating checksums of specified areas. I figured, “Hey, if they can make it work in AutoIT, I’m sure I can make it work in C# even better!”

So using the idea of area checksums, ApocBot can identify what is currently being displayed on the screen and act appropriately.

The basic configuration for ApocBot to run was WindowsXP (Classic Theme) running at 1024×768 (32-bit color depth).

The Classes/Controllers

MTGOController.cs

The MTGOController class holds the primary logic for ApocBot including the main program loop and all MTGO Object definitions.

On start up, all the MTGO Objects are defined. Currently these are all hard coded but I had planned to eventually move them all to an XML file and just have them deserialized on start up (any takers on this? ).. but for now, they’re all hard-coded.

After everything is setup, the main program loop will start. The basic function is this:

1. Grab a screen shot
2. Determine the state of the bot (switch statement)
3. Based upon the state, check the screen for any known events/items to act upon
4. Sleep 250ms (to avoid CPU spike caused by constant screen polling)

ScreenController.cs

The ScreenController class holds all the logic required to grab the screen shots and calculate the area checksums.

MouseKeyboardController.cs

The MouseKeyboardController class holds all the logic required to make mouse movements and send keys to the program. Hooks into the Windows API are used to simulate the mouse events to the OS.

The Objects

There are really three main objects used by ApocBot to know what’s going on:

1. MTGOScreen - Represents a screen in MTGO, such as the “Home” or “Trade” screens (Multiple Areas, Multiple Points)
2. MTGOArea - Represents an area on a screen, such as a pop-up dialogue, card graphic or window (Multiple Points)
3. MTGOPoint - Represents a point on the screen, such as an “OK” or “Cancel” button

Objects are referenced in the system to main properties: Name and Checksum.

The checksums that are currently hard-coded might be out of date. You can generate new checksums using the AreaChecksum() method in the ScreenController.

Almost forgot! It’s named “ApocBot” because Apocalypse was my favorite Magic: The Gathering expansion!

Cheers!

I re-ran some tests using the StopWatch class to hard code some metrics into the application and they also confirmed the findings. What’s up? How could a class that everyone says you can use to your hearts content when it came to string concatenation was failing me?

Turns out, it was a mix of misuse and a common misconception about the StringBuilder Class.

One of the first things you learn while picking up .NET is that the StringBuilder Class is your friend when it comes to concatenating large strings in memory. It beats the pants off of String.Concat and String.Format, while also being a mutable object in the Framework utilizing an in-memory buffer.

I used JetBrains dotTrace to help profile the application and it was very evident from the get-go that StringBuilder was causing the whole process to slow down.

The nature of my application was basically reading in a text buffer 1 character as a time, and using the StringBuilder as an output buffer. So for a 1k file, The method Append(char) would be called 1024 times. A 600k file would call Append(char) 614,400 times.

So why was I getting burned in execution time? The issue turned out to be two fold.

First, there’s overhead cost to the call. I don’t care how lightweight your method is, if you’re calling it SIX HUNDRED THOUSAND TIMES, it’s going to take a bit. Let alone a method who handles a string buffer in memory and string manipulation. So basically, no matter how fast StringBuilder actually is, it’s not a free call and you should consider the fact that the call still has overhead when architecting your solution.

Architecture brings me to my second point. While writing each character individually made sense initally, it seems that it was just lazy The optimized route would have been calling Append with a SUBSTRING of the input buffer, this way we avoid the overhead of multiple calls by writing all the neccisary data in one big blob.

So 600,000 calls to StringBuilder.Append(char) becomes only a few hundred calls to StringBuilder.Append(string.Substring(start, count)). Sure, the Substring Virtual Method itself has overhead, but it’s still less than the thousands of calls to Append(char) that we’re saving ourself

Conclusion?

StringBuilder is fast, but it’s not free. Take this into consideration when utilizing it while appending large data sets in small chunks.

Cheers!

I had a couple goals in mind while creating my routine:

1. KEEP IT SIMPLE — A lot of routines out there WORK, but their code is too overly complicated for their own good. This over complication leads to slowness which brings me to my next goal It should be a simple class that accepts input data, with simple public accessors that are easy to understand even for the novice developer (sorry folks, no asynchronous delegates).

2. MAKE IT FAST — When dealing with large amounts of data in C#, especially when running it through an algorithm, it’s all too easy to use all the handy built in virtual methods or using other build in tools which make coding easier with speed being the sacrifice. Die hard C++ developers will point to these routines as C#’s downfall as a legitimate language when it comes to data intensive tasks.

The class I came up with is pretty simple. I use a Generic List to store a collection of “Leaf” objects, which have several basic attributes that help not only identify its value but also its place in the tree. Using this method, I was able to utilize the built in methods of the List object (I know, for shame…. but it’s easier in this instance) by firing off anonymous delegates for searches and comparators. I’m not terribly worried about using these virtual methods here only because the creation and encoding of the tree is usually the smallest task in the process

The encoding and decoding is where I decided to focus on optimizations since this is where the BULK of the work is done. The .NET Framework has several methods that make working with binary data easy. You can use the Convert.ToString() method which allows you to pass in a BASE option, thus allowing you to convert any character to it’s binary representation. My original implementation used that method and the end result as embarrassingly slow

I went back to the drawing board and thought to myself, “If I had to re-write this in C++, how would I handle the encoding?” Duh, I’d be using bitwise operators up the wazoo!

After some recoding and pulling my hair out for a couple of hours, I was able to re-write the routine using bit operations and it works! On top of all that, it’s fast as all get out! My current benchmarks had it encoding a 1MB data set in under 1 second with ~50% compression. Not too shabby! Of course, compression ratios will vary depending on how normalized the input data set is.

Now I’m sure there’s some question you have on your mind and I’ll try to address them now:

Q: Does it use a lot of memory?

A: You bet your sweet ass it does! Seriously though, it’s only the method in which I setup the class that requires the memory. I establish an input buffer within the class that you can write the ‘raw’ data to, which is then read from during encoding. In addition, during encoding I create an output buffer in memory where the ‘encoded’ data is written. So it stands to reason that if you’re encoding 100MB of data, this routine can easily gobble up 200MB of RAM or more. The rule of thumb I found was File Size * 4 would be the memory requirement. There’s optimizations you can make that would lower the memory footprint (like, build the frequency table without buffering then read the input 1 byte at a time, say from a file), but I felt that would over complicate the solution and make it too focused for one specific instance. The current implementation is kept general for a reason

Q: Could this be done faster in C++?

A: Yes, probably…. but not much faster. Although the code is written in C#, at run time the IL is compiled to x86. The bit operations we’re using would compile the exact same as a C++ routine (XOR is XOR, I don’t care what language you’re using). In addition, the encoding of the data itself is only using primitive native types which limits any cross language differences. In fact, you can paste the encode and decode routines into C++ and they work! Your only speed improvement might be in the generation of the tree itself… but even then, that’s super small overhead when compared with the amount of data you’re probably compressing.

Of course, all that applied to the Encoding (Compression) side of the house, the decompression routine is pretty slow (about 3 seconds per 1MB of decompressed data) and could probably use some more optimizations.

Q: Is this any better than using the built in GZip or Deflate classes available in System.IO.Compression?

A: It’s not even close to being in competition with those routines This is really just a proof of concept for BASIC Huffman Coding, which doesn’t take into account advanced features of modern compression routines such as pattern or content mapping. This routine is slower (especially in decompression), so I wouldn’t go making anything like this your #1 choice for a compression routine if others are available So use this for educational purposes only.

Q: What version of the .NET Framework will this work with?

A: The code here was written in Visual Studio 2008 targeting .NET 3.5. I use some Framework 3.5 specific things (such as object initializers), but nothing that would make conversion difficult. I avoided LINQ only because I’m not entirely sold on the idea and I still like using anonymous delegates. This code could be converted to Framework 2.0 with minor changes and possibly Framework 1.1, but that might require a little more effort.

Q: What is the format used to store the compressed data?

A: I encode the decompression information within the final output stream. The output format is like this:

Bytes 0 – 8: Final Output Size (Not used, but there as a checksum if needed in the future)

Byte 9: Number of Bytes in the Decode Dictionary

Bytes 10 – n: Decode Dictionary

Between the Decode Dictionary and the actual data I add the characters “BCD” (which stands for Binary Coded Data). This lets me know where the dictionary ends and the actual coded data begins. It helped during debugging and I figure it’d help anyone else out there as well while working with this routine

Q: What’s with the essay? Just give me the code!

A: Fine! Seriously though, the only reason I’m doing such a long write-up on the code is to help people who are perhaps beginning to look into this sort of code for the first time and might have questions on why I did things a certain way. Understanding WHY the code was written helps understand how it operates.

So that’s the high and low of it!

Hope this helps someone out and if you have any questions, please feel free to leave a comment!

Cheers!

Huffman.zip – Download (3k)

Sounds simple, right?

So of course now when my wife and I actually sit down to play a DVD movie, randomly it’ll just freeze the picture and appear to lock up. It’s not a hard lock as my XBox still responds to IR commands (open/close the tray, power on/off). At first I suspected the Denon receiver as it is taking the video from the XBox in using component and scaling it out via HDMI. Multiple restarts and power on/offs of the receiver and the picture wouldn’t unfreeze meaning the XBox 360 itself had locked up.

What the hell!

I’m sure there are some who are at this point thinking, “Your XBox is defective!”… I would completely and 100% agree (as I’m on my 3rd one), but I can play titles like Bad Company and Arcade games for hours with no problems, so I doubt it’s a hardware defect within the machine itself.

I suppose I’ll swallow a big ole’ slice of humble pie tonight and call Microsoft Support, BUT, just in case they find nothing wrong I’m curious if anyone else has had this issue!

Thanks!

I recently purchased a MacBook Pro for the sole purpose of developing software for the Mac/iPhone platform. It’s completely unknown to me and currently the XCode IDE looks nothing if not completely confusing compared to the friently intuitive interface of Microsoft Visual Studio that I’m used to

What brought me to this point? Well, two things.

First, I’ve been wanting to dive into Objective-C for a little over a year now and started dabbling with it a while back on my old G4 Powerbook. I was impressed with the coolness of it, but being that I was developing on a G4 Powerbook, I was limited as by this time Apple had begun the mass migration to Intel based systems.

Second is that I want to develop applications for the iPhone. Not to make a mint or anything, more or less because I think I have some good ideas on programs people would use. Like how Apple provided the iTunes controller, well, hows about a WWWinamp controller for the iPhone? Perhaps one that lets you search your library locally on the iPhone without even connecting to WWWinamp?

I started down this path because there’s currently a program for sale on the iTunes App Store that lets you control your instance of WinAmp remotely… for $4.99!!! What the crap?! I was floored that the author would expect that kinda money for a program when there are PLENTY of other FREE alternatives out on the web (like WinAmp Remote, AjaxAmp or WWWinamp). I made it my mission to release a comparable program

It’s just going to take a month or so to ramp up on the new IDE and learn how to do SOAP calls and whatnot. Should be a fun adventure though! My wife begins the final semester of her Teaching Credential program in a couple weeks, so I’ll have four nights a week alone to myself to nerd out and gorge on Hot Pockets!

I have another version of WWWinamp in the works as well. This is basically some code modernization for sections that I wrote over a year ago. Updating things to use Generic Methods and whatnot. You know, fun .NET stuff

Cheers!

I’ve taken the “beta” tag off of it as it seems the recode of the HTTP request processing seemed to work without issue and no bugs were reported (other than the ‘paging’ issue, which really was a defect in the script parser).

Changes for this version are:

• [HTTP] Daemon will now start if a WinAmp instance is not found
• [HTTP] Able to specify max number of HTTP threads to process (Default 25)
• [Script Parser] Fixed “Paging” issue when searching through the Libary
• [Script Parser] Script Files (*.wwa) will be run from the same folder as the file requesting them
• WinAmp Handles will be updated if a new instance is detected

Be sure to use the latest config file included with this release as it has a new key value, WWWinamp.HTTP.ListeningThreads, which tells the HTTP daemon how many threads to use max. This is set to 25 by default if no key is found, but it will throw an error. So if you don’t want to use the packaged configuration file because you’ve hacked together your own, you can manually add this key with any value you’d like.

This value isn’t a huge addition but it can help maintain system stability under heavy load. This way WWWinamp won’t spin up an infinite amount of threads if there’s heavy resource contention or there’s a problem that is causing threads to hang.

Also, the other ‘fixes’ were requests made by? ‘dawolf’ here in my blog so I suggest more people get involved and we can make WWWinamp better!

In addition, if you’ve made your own skin and want to share it, please feel free to contact me and I can post it here to my blog for others to use.

As for future releases, they may be a little slow in the coming month. I’m going to be diving into the iPhone SDK and Objective-C programming using XCode on my Apple. I think a remote media interface would be totally awsome if I can have a native WWWinamp application on the iPhone talk to the WWWinamp server using WCF/SOAP. It’d be way easier than using the webkit crap and I could make it have a lot more functionality (perhaps tuning in to a shoutcast stream VIA the iPhone?). Anyways, just thought I’d give you guys a little bit of information on that.

Cheers!

WWWinamp v4.2 Build 2990 – Download (56k)

It was then announced that the official iPhone SDK which runs natively on the iPhone will be released later this month. So soon you’ll have a host of local applications which will most likely include 3rd party media players. It then dawned on me, if a user could use WWWinamp from the iPhone with an interface that interacted nicely on the iPhone, then someone’s iPhone could act as a remote control from ANYWHERE allowing them access to their home media library.

Well, this is just a heads up that I’m currently working on a skin for WWWinamp that will let it work on the iPhone (better)! Soon you all will be able to control your media center or home copy of WWWinamp from anywhere using your Apple iPhone!

I’m still working on the organizing the data on the screen in a manner that makes sense and is easy to use. All the functionality that comes with the default skin will be included in the iPhone version, including cover art and all the Winamp Playlist functions.

It’s actually pretty sweet controlling Winamp from an Apple iPhone and I hope I can polish this up and put out a beta version this weekend. I’m still trying to fish up some resources on the WebKit API and the WebKit specific tags to make things integrate nicely.

Keep your browsers posted here for updates in the coming days!

I’m currently working on the latest version of WWWinamp after taking another hiatus from the project in order to clear my head and work on some other fun things such as the previously mentioned XNA projects

The next version of WWWinamp will be targeted for the .NET 3.5 Framework and you will need to have it installed on your machine in order to run WWWinamp. Most of the changes in this version are behind the scenes and will probably be transparent to you, the user. BUT, some visible changes in the upcoming version of WWWinamp 4.2 will be:

• Automatic Version Checking for new releases of WWWinamp
• Ability to enable or disable Automatic Version Checking
• Ability to enable or disable Windows Vista warning message
  
• Improved performance under load

I know, nothing earth shattering on the horizon. “Why is this?” you might as. Well, the honest reason is that I haven’t received any bug reports with issues that weren’t related to a misconfiguration or user error. This is a good thing! It means that WWWinamp is stable and there aren’t any horrible bugs that continue to nag users. The downside to this is that because WWWinamp is pretty feature rich as is, I’m running low on ideas that could be implemented.

An idea I’ve tossed about and am thinking about trying out is creating a WWWinamp .NET assembly for user within your own Windows or ASP.NET applications developed using the .NET framework. The assembly would handle the media library and searching of the library so you wouldn’t have to code your own database and file searching routines, as well as handling the unmanaged code required to interface with Winamp (if it’s running locally on the WWWinamp machine) or communicating via WCF to a remote instance of WWWinamp running the WCF daemon.

I think that between the WWWinamp Server and my proposed WWWinamp Assembly, that’d give both advanced and novice users full control over their setup.

As I mentioned in a previous post, I’ve moved WWWinamp into Visual Studio 2008 with Team Foundation Server. Within the latest version of Visual Studio 2008 there is a set of code analysis tools which calculate such items as Code Maintainability and Cyclomatic Complexity. I ran it on WWWinamp and it scored the following:

Maintainability Index: 83 (out of 100)

Cyclomatic Complexity: 636

Depth of Inheritance: 7

Class Coupling: 151

Lines of Code: 1,829 (!!)

After over a year, I think WWWinamp has gotten better and continues to improve with every version!

Big thanks to everyone for their support and keep your eyes peeled here for the latest version of WWWinamp!

After fumbling around with that for an hour or so an only succeeding in creating a small square on the screen, I headed over to Microsoft.com to see if any MSDN articles might exist to help me along in my ‘ground up’ learning of XNA. I was pleasantly surprised when I found a great article titled “Your First Game: Microsoft XNA Game Studio in 2D“. This was EXACTLY what I was looking for as it starts from the ground up, assuming the reader has never done game programming before, let alone 3D game programming.

The example provided my Microsoft in this article is a simple 2D Texture of a cat that bounces around the window. I was so pleased with the ease of coding this, I thought to myself, “Heck, how hard could it be to recreate Pong?” So I set upon my task.

Several hours and many Coca-Cola bottles later I had not only my first XNA game running in Windows, but after purchasing the XNA Creators Club annual subscription from the XBox Live! marketplace for $99, I had it running on my XBox 360 as well I decided to take a little extra time and add a debug information screen as well as a small welcome screen

Controls (PC):

Up/Down for Player 1 Paddle (Left): Q/A

Up/Down for Player 2 Paddle (Right): Up Arrow/Down Arrow

Debug Information: F1

Controls (XBox 360):

Up/Down for Player 1 Paddle (Left): Left Thumbstick on Player 1 Remote

Up/Down for Player 2 Paddle (Right): Left Thumbstick on Player 2 Remote

I did run into a couple of ‘gotchas!’ while working with Game Studio. The major one I had trouble with was when you’re developing for the XBox 360 you have to account for overscan on the Television and have your game render within the ‘Safe Area‘. I noticed that when I was playing my Pong! game on my LCD TV at 720p, the edges of the game were cut off and it looked like it was stretched past the borders of my TV. After asking the fine folks in #XNA on IRC about this issue, they were able to help me out. Now I have my Pong! game account for this by setting up an XBox 360 macro which pads the edges of the play area by 50 pixels.

I’m including the Source Code for my version of Pong! which I’ve titled, “2D WndrPong!”. You can work in XNA Game Studio for free using Visual C# 2005 Express Edition along with XNA Game Studio 2.0 To deploy your games to an XBox 360, you must pair your XBox with your PC (using the “XNA Game Studio Device Center” tool) then you must purchase an XNA Creators Club Membership from the XBox 360 Marketplace. I believe the prices are $49.99 for 3 Months, $99.99 for a year. I opted for the entire year since I know it’s going to take me some time to learn

Baby Steps

2D WndrPong! Source Code – Download (36k)

I’ve spent the last week or so playing around with the .NET Framework v3.5 and I really like what I see. Microsoft has also put a lot of work into Team Foundation Server 2008 as well. I’ve setup a Virtual Machine using Virtual PC here running Team Foundation Server 2008 on Windows 2003 Server R2 and was able to get WWWinamp imported + converted without issue!

[zoomer]66|450|0|WWWinamp in Team Foundation Server 2008|1|0[/zoomer]

One thing I have noticed is that TFS does take up quite a bit of resources. I had to kick my VM up to 2GB of RAM in order for it to handle and build WWWinamp without hitting the swap file. Something to keep in mind moving forward I suppose. I’d imagine a large installation of TFS would require something along the lines of a Quad Core CPU with at least 4GB RAM.

There are several features in .NET 3.5 that I plan on implementing in WWWinamp right off the bat:

• Object Initializers – This will really help clean up the already massive code base. It’ll also speed things up a bit
• Embedded Manifest – A feature I’m surprised they didn’t implement with the release of .NET 3.0. Now the .manifest file will be embedded into WWWinamp, so you won’t have the extra file handing around if you run Windows Vista
• Extension Methods – Again, this will help clean up the code base and make things faster. I imagine it’ll take up a little more memory, but these days RAM is abundant

Now, I see LinQ and I’m trying to figure out it’s role in everything. I think it’ll make DB communication easier but I’m not a fan of the anonymous data types and implicit local variables. I think it could lead to some sloppy coding methodology if not kept in check.

I think it’s handy that you can now run a SQL like statement on a generic, but I’m wondering what sets this apart from the FindAll method that already exists in System.Collection.Generics. It does make it easier though to search through a Generic Collection for a developer who doesn’t have a solid grasp on delegates.

So, all that said, things are moving along. I’ll keep you all updated on the progress and please, send in those feature requests!