Monthly Archives: March 2010

Sometimes less is more.

Posted on by
6

 

The more they overthink the plumbing, the easier it is to stop up the drain.

-Star Trek III

Well as of late we’ve been fighting this aging SQL server at work. It was originally a NT 4.0 server with SQL 6.5 upgraded to 7.0 then 2000 with Windows 2000. It also had to do some work with Oracle, and the dba’s were using MSDTC to make sure their transactions were getting committed into Oracle. Oh and to keep it ‘fresh’ the Oracle client was version 7 as it originally was talking to an Oracle 7 DB.

Well as the years go by, that Oracle 7 DB became Oracle 9i (already obsolete!), and we suddenly hit a transaction wall.

And along the way we virtualized the server to go into our VMWare ESX 2.x cluster, and it’s been since migrated to VMWare ESX 3.5

The server was dropping tens of thousands of these XA???????????????????.trc files, into the \winnt\system32 directory.

Well naturally you’ll eventually hit this wall of how many 8.3 translations you can do before the system CRAWLS. And boy oh boy did we hit that wall. So at first my idea was to delete all of these trc files, and let it live, but that’s not such a ‘great’ idea… As this reeks of a fundamental problem.

So the ‘first’ step in all of this madness was to up the OS to Windows 2003 enterprise (it was 2000 Advanced server before) And see how things were doing. The OS upgrade went smoothly I had slipstreamed sp2 into the update, so I only had some 90+ updates needing to be done once the OS had been upgraded. And for ‘good’ measure I thought I’d take the server from 768MB of ram up to 2GB, and set VMWare to allow 4 cpu’s for the database server. The node it was running on wasn’t doing terribly much so what the hell right?

Wrong…

The server was now performing markibly SLOWER… And yes, still dropping TRC files like there was no tomorrow.

After a bunch of digging around, I found out that in 2003 you have to click a box in the component manager to allow XA (cross architecture) transactions! Well now it wasn’t dropping as many XA trc files, but after watching it for a while, when two went to run at the same time, the SQL server would crash with a hex code saying it was out of memory.

Out of memory? I’d just given it more!?

So I did the ‘logical’ thing and gave the system 5 GB of ram, and enabled the /3GB flag in boot.ini

Same crashes.

I moved SQL server up to using 2GB of ram (out of 5, sure why not?). Same error.

Well this sucked, so we tried to update the Oracle client from 7 to 9i. In the process I found I couldn’t un-install the 7 client, nor could the 9i thing just ‘upgrade’ it, 9i kind of installed in parallel. Which led us to our next major fault, after swapping out the new client, removing the Oracle product key from the registry, and re-linking the Oracle servers using a new registration string, now ALL of our transactions against the oracle servers were failing.

Thank goodness for google, as we were able to deduce that the registry key HKEY_LOCAL_MACHINE\SOFTWARE\Microsfot\MSDTC\MTxOCI was populated with all the old Oracle 7 values….

OracleOciLib -> oci.dll
OracleSqlLib -> orasql9.dll
OracleTraceFilePath -> c:\oratrace
OracleXaLib -> oraclient9.dll

So changing it to reflect Oracle 9i and suddenly our transactions were running! Even two at a time!!

But there was no doubt about it the transactions were slower then hell. We had gone from 1 minute to 11 minutes on one, and 5 minutes to just under an HOUR.

I added MORE memory to only find the SQL server couldn’t see the network card. So I added another one, and it got even SLOWER.

So in a minute of panic, I reduced the ram back to 768MB, took the VM from 4 cpu’s to 2 cpu’s and forced SQL server to use a single processor.

And our timings are now fantastic! That 1 minute process can complete now in 12 seconds!!! The other process finishes in about a minute give or take, but it’s tremendously faster.

From what I can gather, since SQL is so IO bound the more ‘top’ hardware you give it, the harder it pushes the IO stalling itself… Naturally it’s different on a physical machine, but sometimes it’s interesting to see what happens.

And may this be a lesson, just because it can emulate multiple CPU’s doesn’t mean it’ll run parallel things ‘better’…

Xenix, K&R and f2c…

Posted on by
4

Well a friend of mine let me have access to a Xenix 386 machine with a C compiler! Which was great, The first thing I built was gzip, as I couldn’t imagine a machine without it…

then I spent the better part of 3 hours ‘fixing’ f2c/libf2c for some Zork fun. Well let me say that while it was fun and all, the C environment from Xenix is…. old. No stdlib.h.. Among other things. Also gzip and f2c had issues where some programs would use ‘local’ copies of procedures… Naturally whatever compiler (MS?) my friend had wouldn’t have any of that!

Not to mention the other slight stumbling block, was that this was a K&R Compiler… Ouch.

However I was able to bash together a version of f2c that ran!

However it left me fixing the prototypes for the output… Because by default f2c outputs in ANSI.

It’s no biggie for ‘hello’ or even the textcnv program for Dungeon, but there was no way I was going to do this for the 100’s of procedures in Dungeon…

But the upshot is that the simple answer was a man page away…

Use the -K flag, instead of the -C flag, and all will be well.

I didn’t bother trying to clean up the gettim.C thing, maybe some other day, but I can’t promise much.

But in the meantime, here it is, for the 2 or 3 people with some kind of legacy Xenix thing out there..

 

Dungeon-2.5.6 for Xenix.

This is an 80386 exe, so you 80286 users would be out of luck… I don’t know if I even can make 80286 stuff.. I think the next thing I’ll have to see how hard it is to build is that ACK as it says it supports Xenix… And free compilers are always ‘good’ things.

A quick follow up to building DLL’s

Posted on by
1

Another slight issue I’ve found with the SLiRP code, is that it expects to be able to call two functions (slirp_can_output and slirp_output). The snag here is that DLL’s expect to be SELF CONTAINED… So how can you have a DLL that needs to call functions from your program?

Void pointers!

So let’s make this more… involved.

dll.c

void (*bob)(int xy);

__declspec(dllexport) int hi(void)
{
(*bob)(12);
return 3;
}

__declspec(dllexport) void Register(void *p)
{
bob=p;
}

test.c

extern void Register(void *p);

void XX(int xy)
{
printf(“XX %d\n”,xy);
}

void main(void)
{
int j;
Register(XX);
j=hi();

printf(“%d\n”,j);
}

And let’s build it…

c:\temp\dll>cl /c /LD dll.c
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.21022.08 for 80×86
Copyright (C) Microsoft Corporation. All rights reserved.

dll.c

c:\temp\dll>link /DLL dll.obj
Microsoft (R) Incremental Linker Version 9.00.21022.08
Copyright (C) Microsoft Corporation. All rights reserved.

Creating library dll.lib and object dll.exp

c:\temp\dll>cl test.c dll.lib
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.21022.08 for 80×86
Copyright (C) Microsoft Corporation. All rights reserved.

test.c
Microsoft (R) Incremental Linker Version 9.00.21022.08
Copyright (C) Microsoft Corporation. All rights reserved.

/out:test.exe
test.obj
dll.lib

c:\temp\dll>test
XX 12
3

So as we can see, the main procedure runs the ‘register’ procedure to tell the dll that the ‘bob’ procedure will infact be XX. Now when the dll invokes it’s ‘hi’ function it can call the XX function from the exe. Now if I were a better programer, I’d use the DLLMain code to make sure that all my virtual void functions cause some kind of ‘readable’ panic message when called so you don’t get the default reaper… Because naturally if you run the dll code without the register function it’ll crash in a not so great manner.

But I’ll leave that as an exercise to the reader.

I’ve made some good progress on the SIMH thing, Ive got the non networked versions & the libpcap stuff building with Visual C++ 2008… I was thinking about using that 2010 RC but that just seems wrong…. But for now all I’ll have to do is apply this logic to the slirp dll, and get building the VC version for linking, and the Mingw version for actual operation….

Build a DLL

Posted on by
Reply

Well this is leading up to a new release of my SIMH projects. The issue that I’ve had with the SLiRP code (usermode NAT) is that it only works when built with GCC, and ONLY when you have no optimizations. Naturally you will loose all optimizations with SIMH if you build the entire project like that. And of course you cannot use Visual C++ to build SIMH, because while it’s faster they use different object files.

So while thinking about what a bummer it is, and how to try to debug SLiRP with Visual Studio, today I had a ‘better’ idea. Well ‘better’ in that I can do this way quicker.

Instead I’ll build SLiRP as a DLL, and have Visual C++ call that!

Ok, now that sounds crazy, but the first thing I’d need is a simple ‘test’ case. First let’s build a dll in MinGW. It took a bit of googling but then I found this super simple example.

dll.c

__declspec(dllexport) int hi(void)
{
return 3;
}

Ok, it doesn’t do that much, but you get the idea. Now we have to compile this with MinGW.

gcc -c dll.c
gcc -shared -o test.dll dll.o

Ok, now this will first compile the C file into an object file, then the linker will set it up as a DLL. Notice that I’m not building the ‘.a’ export file for this DLL. Visual C++ wouldn’t like it anyways, so I don’t need it.

Next, using Visual C++ (I would *assume* just about every version can do this..) I re-build the DLL to create the export library. Yes I know this is weird, but it is the quickest way I know to do this.

cl /c /LD dll.c
link /DLL dll.obj

Cool, now we’ve built dll.c as a DLL under both MinGW & Visual C++. Now for a ‘test’ program to drive our ‘staticly linked’ dynamic link library.

test.c

void main(void)
{
int j=hi();
printf(“%d\n”,j);
}

Ok, now with that out of the way, let’s compile & link!

cl test.c dll.lib

Cool, now we’ll have a test.exe!

Let’s run it!

c:\msys\1.0\src\dlltest>cl test.c dll.lib
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.21022.08 for 80×86
Copyright (C) Microsoft Corporation. All rights reserved.

test.c
Microsoft (R) Incremental Linker Version 9.00.21022.08
Copyright (C) Microsoft Corporation. All rights reserved.

/out:test.exe
test.obj
dll.lib

c:\msys\1.0\src\dlltest>test
3

Wow that was exciting! Now for the people paying attention, you’ll remember that the test.exe was linked against dll.lib which was built with the Microsoft compiler, and is calling the Microsoft generated dll.dll. Yes you are correct. But always test with a known positive, before you throw in the negative I say…

c:\msys\1.0\src\dlltest>move test.dll dll.dll
Overwrite c:\msys\1.0\src\dlltest\dll.dll? (Yes/No/All): y
1 file(s) moved.

c:\msys\1.0\src\dlltest>test
3

Holy crap! It worked!!!!

That’s right, now I have built a very simple test program that’ll call a DLL function built with MinGW!

And that, is how you can get around a kooky problem where some programs build with one version of a compiler tweaked one way, and you want to use those bits in another program built another way.

Just ask any language refugee from the 1980’s how hard this was to do on the PC platform.. While VMS had a common library spec for all languages, not every platform (MS-DOS,UNIX) was as lucky. But with the rise of things like Windows with DLL’s, and of course NT thanks to the VMS crew it’s in modern systems today.

Let me tell you, linking Quick Basic to C was… not all that much fun. And Turbo C++ was right out!

Anyways I’ll leave you with this much, and at the worst case, much like making bootable CD’s, it’s a good note for me to find this stuff later when I need it.

SHOWSTOPPER!

Posted on by
5

show-stopper-coverI was browsing around at a book store, and I came across the book “SHOWSTOPPER” the breakneck race to create Windows NT and the next generation at Microsoft.

If you have ever lived through the Windows NT 3.x days you’ll find this a very interesting read. It goes into the big personalities, and of course covers the working habits of Dave Cutler… Although it does paint him in some really odd colors, mostly as an antisocial kind of dictator pushing people to produce the largest program Microsoft had ever produced at the time.

But there is no doubt, Cutler could not have written Windows NT at Digital, as DEC was too fond of hardware lockins (look at VMS & Ultrix/True64). And it does cover the major animosity of Cutler towards DEC with the cancellation of the Prisim/Mica projects, and then the later “I told you so” moment when DEC licensed Windows NT from Microsoft (although other reports claim that DEC threatened MS with a lawsuit, and MS gave them access to NT, along with some money…). Apparently the mantra was “Dec could have had NT for free”..

There is also coverage of the culture clash of what happened when Microsoft had absorbed the Prisim & Mica engineering teams from DEC, and how they did not get along with Microsoft staff, and even did their best to poke holes in the current offerings of MS-DOS & OS/2 as either a toy, or a joke.

One thing I found interesting, is that the book mentions the WLO project, as the foundation for what would be the ‘Win32’ system. WLO if you remember was a port of the Windows Libraries to OS/2. It was very interesting in that Windows, OS/2 and even MS-DOS & Win16 via WOW were all not part of the main Windows NT group, but rather ‘tacked on’.

However it is quite interesting that the design decisions made for a very portable and modular operating system, that survived it’s original CPU & platform being changed 1/4th the way through development, and then the removal of the primary API.

Another thing that was interesting was some of the ‘fixes’ for the too slow, too big that would plague the early versions of NT, was the idea of demand paging portions of the kernel.. I for one would go insane with the blue screens about paging non page-able areas or some other VM error… But the truth was NT was written by people who came from a minicomputer world, and as the book made evident from time to time, they did NOT use PC’s.

Needless to say, the book was somewhat spot on, in that it’d take 10+ years for computers to catch up to what Windows NT was written for. I for one can remember trying to run this on a 386sx-16 and it was horrible… But if you install it on a Pentium II the 3.x series simply FLIES… And in emulation on modern machines it has incredible performance.

While Windows NT 3.1 was no doubt a 1.0 release, 3.5 was a 2.0. The x86 optimizations really payed off, and kicking out the Spider TCP/IP stack, and bringing in the new MS stack helped a LOT. There is no doubt back in 1994 as SLIP & PPP accounts were becoming more common place, Windows NT 3.5’s networking was the easiest to configure and use. Linux back then really was in it’s infancy, and the dialup scripts for pap/chap/pppd were… a nightmare.

“Dogfooding” was another interesting, and necessary thing as once NT was able to start running programs it was important to make people start using it as quickly as possible to shake out bugs in the system. Its also interesting to note the reluctance of the kernel team to deal with the graphical part of NT, and how the first versions were text only. Another weird part was how the security in Windows NT was an after effect, of the internal networking group cooking up what eventually became the domain & trust model. Not to mention how NTFS almost didn’t make it because the filesystem people (all two of them!) were so busy making sure HPFS worked correctly.

There is no doubt that such a ‘ground up’ OS of this magnitude hasn’t been attempted since 1988. It took Microsoft 5 years to get Windows NT out the door, but there is no doubt looking around in the year 2010, Windows NT has a long life ahead of it.

For those interested, you can find it on amazon.

So I was looking in the UK for ACORN

Posted on by
Reply

and other old kit… I came up with nothing..

Does anyone know any stores, that sell ancient stuff, like megadrives, acorns, BBC micros etc?

There must be somewhere in the British Isles….

Also I’m wondering if I should check prices for Russia in the Summer, or Japan in the Winter……

I think I need a vacation from this vacation.