Mercurial > sdl-ios-xcode
annotate test/automated/README @ 4447:947201caa46e
Added automated test to Xcode project plus needed changes to SDL_RWFromFile to be OS X bundle aware.
The Mac OS X project has a new target called testsdl which builds the automated test. I looked at using Xcode's native unit test support, but the impedance mismatch between the existing automated test structure and Apple's was more than I could handle.
As such, the testsdl application is a full blown proper OS X application, which means it is a self-contained .app bundle. This immediately revealed some problems from the automated test. The largest problem was the assumption about the current working directory and where to find resources. (I suspect Windows may have a similar problem depending on circumstance.) To open resources, the test was looking in directories relative to the SDL source directory, but this will not work well with self-contained .app bundles and Xcode which can place its built applications almost anywhere. And for real-world situations, this is pretty useless anyway.
So I modified SDL_RWFromFile to do special things on OS X. First, it will look for a file in the .app bundle. If not found, it will fallback and just call fopen as it used to do.
I also had to modify the automated test itself because it had a contrieved test which called fopen directly to do read from an existing FILE pointer. In addition, there was a write test. Since a .app bundle is likely going to be read-only, I added a special case for OS X to write to NSTemporaryDirectory.
I expect these changes should work for both Mac and iPhone OS (which includes iPad).
I will update the iPhone Xcode project next.
Finally, FYI, the X11 automated test seems to be failing. Below is my output.
Pending breakpoint 4 - "-[NSException raise]" resolved
Platform : All tests successful (2)
SDL_RWops : All tests successful (5)
Rect : All tests successful (1)
SDL_Surface : All tests successful (6)
Rendering with cocoa driver : All tests successful (3)
Assert Failed!
Blit output not the same.
Test Case 'Renderer x11'
Test Suite 'Rendering with x11 driver'
Last SDL error ''
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSGetSurfaceBounds
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorFailure: Set a breakpoint @ CGErrorBreakpoint() to catch errors as they are logged.
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSBindSurface: Invalid window 0xa150
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSBindSurface: Invalid window 0xa150
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSBindSurface: Invalid window 0xa150
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSGetWindowBounds
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSGetSurfaceBounds
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSBindSurface: Invalid window 0xa150
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSBindSurface: Invalid window 0xa150
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSBindSurface: Invalid window 0xa150
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSGetWindowBounds
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSGetSurfaceBounds
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSBindSurface: Invalid window 0xa150
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSBindSurface: Invalid window 0xa150
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSBindSurface: Invalid window 0xa150
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSGetWindowBounds
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSGetSurfaceBounds
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSBindSurface: Invalid window 0xa150
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSBindSurface: Invalid window 0xa150
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSBindSurface: Invalid window 0xa150
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSGetWindowBounds
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSGetSurfaceBounds
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSBindSurface: Invalid window 0xa150
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSBindSurface: Invalid window 0xa150
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSBindSurface: Invalid window 0xa150
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSGetWindowBounds
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSGetSurfaceBounds
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSBindSurface: Invalid window 0xa150
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSBindSurface: Invalid window 0xa150
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSBindSurface: Invalid window 0xa150
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSGetWindowBounds
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSGetSurfaceBounds
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSBindSurface: Invalid window 0xa150
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSBindSurface: Invalid window 0xa150
Sat May 8 00:30:34 iMacAL.local testsdl[71586] <Error>: kCGErrorIllegalArgument: CGSBindSurface: Invalid window 0xa150
Rendering with x11 driver : Failed 1 out of 4 testcases!
Rendering with dummy driver : All tests successful (3)
SDL_Audio : All tests successful (1)
Tests run with SDL 1.3.0 revision 1095906
System is running Mac OS X and is little endian
author | Eric Wing <ewing . public |-at-| gmail . com> |
---|---|
date | Sat, 08 May 2010 00:54:22 -0700 |
parents | 22ac66da0765 |
children |
rev | line source |
---|---|
3259
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
1 |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
2 |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
3 SDL Automated Testing Framework User Documentation |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
4 by Edgar Simo Serra |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
5 |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
6 |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
7 |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
8 Abstract |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
9 |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
10 The SDL Automated Testing Framework, hereby after called SDL_AT, is a meant |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
11 to test the SDL code for regressions and other possible failures. It can also |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
12 be used to display what your SDL set up supports. |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
13 |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
14 |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
15 |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
16 Basics |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
17 |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
18 The main way to use the framework is to compile it and run it, that can be |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
19 done with the following command: |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
20 |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
21 $> make test |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
22 |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
23 It should then display something like: |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
24 |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
25 Platform : All tests successful (2) |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
26 SDL_RWops : All tests successful (5) |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
27 SDL_Surface : All tests successful (6) |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
28 Rendering with x11 driver : All tests successful (4) |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
29 |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
30 Indicating that all tests were successful. If however a test fails output it |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
31 will report the failure to stderr indicating where and why it happened. This |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
32 output can then be sent to the developers so they can attempt to fix the |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
33 problem. |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
34 |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
35 |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
36 |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
37 Advanced |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
38 |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
39 By passing the "-h" or "--help" parameter to testsdl you can get an overview |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
40 of all the possible options you can set to furthur tweak the testing. A sample |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
41 of the options would be the following: |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
42 |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
43 Usage: ./testsdl [OPTIONS] |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
44 Options are: |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
45 -m, --manual enables tests that require user interaction |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
46 --noplatform do not run the platform tests |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
47 --norwops do not run the rwops tests |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
48 --nosurface do not run the surface tests |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
49 --norender do not run the render tests |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
50 -v, --verbose increases verbosity level by 1 for each -v |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
51 -q, --quiet only displays errors |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
52 -h, --help display this message and exit |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
53 |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
54 |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
55 |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
56 Developers |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
57 |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
58 See SDL_at.h for developer information. |
22ac66da0765
Merged Edgar's code changes from Google Summer of Code 2009
Sam Lantinga <slouken@libsdl.org>
parents:
diff
changeset
|
59 |