[PORTS] Multiple Crashs on OSX Intel

[Marc Simonin]

Hello !

Here is the problem :


We have a database that works perfectly on a Xserv G5, 10.4.8. (PPC) , but
encounter multiple postmaster and postgres crashes when we try it on an
intel mac.

We tried on a 10.4.8 intel xserv xeon, and on a 10.4.8 intel macbook pro,
with postgres8.1.5 and 8.2.1 ... with the same issues.

The postmaster is well launched, and no problem is logged.
But after a few times, when the machine begin to have many requests, i can
see almost regular crashes from some postgres or postmaster launched
process.

Postgres log is a bit helpless cause crashes seems to occure "randomly" with
every requests, even with the autovacuum.


The CrashLog log for every crashes one of this two codes

Exception: EXC_BAD_ACCESS (0x0001)
Codes: KERN_PROTECTION_FAILURE (0x0002) at 0x9003b7e0

Exception: EXC_BAD_ACCESS (0x0001)
Codes: KERN_PROTECTION_FAILURE (0x0002) at 0x00000000

I was thinking of some memory problem, so I tried to run the server without
any special configuration in postgresql.conf like shared_buffer (exept
allowing external connections) but, the problem still remains.


I don't know why it works on ppc and not on intel. Must I try to recompile
postgres from the sources with specials options ?

Does anyone have an idea or a hint ?
Thank you in advance !


Marc



PS : This actually occured on these platforms :

CPU : MacbookPro Core2duo & Xserve Xeon (both Intel cpus)
OS Version: 10.4.8 (Build 8N1051) & (Build 8N1215)
Postgres Version : 8.1.5 (Macport & Entropy ports) , 8.2.1 (Entropy port)




---------------------------(end of broadcast)---------------------------
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

[Tom Lane]

Post by Marc Simonin
The CrashLog log for every crashes one of this two codes
Exception: EXC_BAD_ACCESS (0x0001)
Codes: KERN_PROTECTION_FAILURE (0x0002) at 0x9003b7e0
Exception: EXC_BAD_ACCESS (0x0001)
Codes: KERN_PROTECTION_FAILURE (0x0002) at 0x00000000

Always those same addresses? If so I'd wonder about a corrupt-data
problem. Can you get a stack trace from the core files?

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

[Tom Lane]

But I put here one of the Crash logs and put attached the gzipped entire
Crashlog file (if it pass through the mailing list !).

Wow, those stack traces are all over the map, aren't they. Either you
are hitting a dozen different Postgres bugs that no one else has ever
seen, or you've got a flaky machine. I think the second is considerably
more likely --- especially since several of the crashes are in startup
code that every backend process ought to execute exactly the same way
every time.

Perhaps bad RAM, or a bad motherboard? I've also seen machines go nuts
like this if the fan froze up, allowing the CPU to overheat. Anyway,
take it back to Apple ... I hope it's still under warranty ...

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

[Tom Lane]

Tom, it is very unlikely that the issue is located with the hardware as we
tested on 2 brand new hardware and both exibit exactly the same symptoms
despite they are differents models...

[ shrug... ] It's not impossible that you've got two lemons ... stranger
things have happened. One pretty obvious opportunity for a common-mode
failure is if you loaded them up with RAM chips from the same batch.

The symptoms shown in your crashreporter logs don't look anything like
a software problem to me: they're not consistent, and a lot of the
crashes are in code that is exercised exactly the same way on every
process start. Also, we're not seeing reports of similar problems from
anyone else running PG on Intel Mac; which is definitely a nonempty
population --- there's one in the buildfarm for instance, and it's
showing zero failures in the back branches:
http://buildfarm.postgresql.org/cgi-bin/show_history.pl?nm=jackal&br=REL8_1_STABLE

So I'm going to stick to my bet that it's a hardware problem.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

[Fabrice Vincent]

Hi,

Marc is unavailable today so I take over in order to move forward with our
crash issue.

Tom, it is very unlikely that the issue is located with the hardware as we
tested on 2 brand new hardware and both exibit exactly the same symptoms
despite they are differents models...

Would you have any other idea of where to look for the cause of these crash?
For example would it be possible that the crash would be caused by some
system library rather than the postgres code itself?
Also, is there any debugging option we could turn on on the faulty systems
in order to pin point where is located the bug?

Thanks a million for your help.

Best regards.
Fabrice

Date=A0: Thu, 01 Feb 2007 12:29:01 -0500
Objet=A0: Re: [PORTS] Multiple Crashs on OSX Intel
=20

But I put here one of the Crash logs and put attached the gzipped entire
Crashlog file (if it pass through the mailing list !).

=20
Wow, those stack traces are all over the map, aren't they. Either you
are hitting a dozen different Postgres bugs that no one else has ever
seen, or you've got a flaky machine. I think the second is considerably
more likely --- especially since several of the crashes are in startup
code that every backend process ought to execute exactly the same way
every time.
=20
Perhaps bad RAM, or a bad motherboard? I've also seen machines go nuts
like this if the fan froze up, allowing the CPU to overheat. Anyway,
take it back to Apple ... I hope it's still under warranty ...
=20
regards, tom lane
=20
---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend