Folks, it's Friday and I have an anecdote to share before I return to today's coding fiasco. I'll just tell you what happened and try to avoid making judgements, I'll leave that to you. For about 4 years we've had an Azure hosted Web API/service driving a moderately complex Blazor app and some other smaller clients. The service hosted a C++ library that did the heavy lifting of generating cross-tabulation reports. The trouble was, that the service would randomly crash deep inside the C++ dll and it would leave no useful diagnostic evidence, usually just a hint about some kind of memory access violation. It would never crash in testing, only in Azure. I presume here is a way to diagnose this sort of crash in Azure hosting, but you probably need the minidump and symbol files out of the C++ compile, I'm not exactly sure, as I just couldn't face the toil, and by great luck it was due for replacement anyway. Some very large US companies were suffering from the crash interruptions and there was a serious risk that we could lose their business. The lucky part is that the huge C++ codebase was already being rewritten in C#, so we went into a frenzy of continued conversion and testing, and the C# replacement is now about 90% rolled-out. and guess what?! ... The random crashes are gone and one customer even sent us a message of thanks for the new reliability. We did have a few small unhandled exceptions, but I simply went to the Azure portal logs and the stack trace pointed us straight to the problem point. We could usually publish a fix within half an hour. So years of random C++ crashes were completely cured by a C# rewrite. *Greg K*
I have never understood the fixation with C++ unless you're in the business of writing kernels, device drivers, embedded systems, etc. On Fri, 1 Sept 2023 at 08:44, Greg Keogh via ozdotnet <ozdotnet@ozdotnet.com> wrote:
Folks, it's Friday and I have an anecdote to share before I return to today's coding fiasco. I'll just tell you what happened and try to avoid making judgements, I'll leave that to you.
For about 4 years we've had an Azure hosted Web API/service driving a moderately complex Blazor app and some other smaller clients. The service hosted a C++ library that did the heavy lifting of generating cross-tabulation reports. The trouble was, that the service would randomly crash deep inside the C++ dll and it would leave no useful diagnostic evidence, usually just a hint about some kind of memory access violation. It would never crash in testing, only in Azure. I presume here is a way to diagnose this sort of crash in Azure hosting, but you probably need the minidump and symbol files out of the C++ compile, I'm not exactly sure, as I just couldn't face the toil, and by great luck it was due for replacement anyway. Some very large US companies were suffering from the crash interruptions and there was a serious risk that we could lose their business.
The lucky part is that the huge C++ codebase was already being rewritten in C#, so we went into a frenzy of continued conversion and testing, and the C# replacement is now about 90% rolled-out. and guess what?! ... The random crashes are gone and one customer even sent us a message of thanks for the new reliability.
We did have a few small unhandled exceptions, but I simply went to the Azure portal logs and the stack trace pointed us straight to the problem point. We could usually publish a fix within half an hour.
So years of random C++ crashes were completely cured by a C# rewrite.
*Greg K* -- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
I have never understood the fixation with C++ unless you're in the business of writing kernels, device drivers, embedded systems, etc.
The library we're phasing out was started around 1997, so you can throw the authors a bone because the world was very different back then. Your main choices were C/C++, VB and Java. I stand by my opinion that C++ is the most absurdly complex and idiotic language in contemporary use. Ooops! I made a judgement. *GK*
On Fri, 1 Sept 2023 at 08:44, Greg Keogh via ozdotnet < ozdotnet@ozdotnet.com> wrote:
Folks, it's Friday and I have an anecdote to share before I return to today's coding fiasco. I'll just tell you what happened and try to avoid making judgements, I'll leave that to you.
For about 4 years we've had an Azure hosted Web API/service driving a moderately complex Blazor app and some other smaller clients. The service hosted a C++ library that did the heavy lifting of generating cross-tabulation reports. The trouble was, that the service would randomly crash deep inside the C++ dll and it would leave no useful diagnostic evidence, usually just a hint about some kind of memory access violation. It would never crash in testing, only in Azure. I presume here is a way to diagnose this sort of crash in Azure hosting, but you probably need the minidump and symbol files out of the C++ compile, I'm not exactly sure, as I just couldn't face the toil, and by great luck it was due for replacement anyway. Some very large US companies were suffering from the crash interruptions and there was a serious risk that we could lose their business.
The lucky part is that the huge C++ codebase was already being rewritten in C#, so we went into a frenzy of continued conversion and testing, and the C# replacement is now about 90% rolled-out. and guess what?! ... The random crashes are gone and one customer even sent us a message of thanks for the new reliability.
We did have a few small unhandled exceptions, but I simply went to the Azure portal logs and the stack trace pointed us straight to the problem point. We could usually publish a fix within half an hour.
So years of random C++ crashes were completely cured by a C# rewrite.
*Greg K* -- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
-- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
Back then C++ wasn't so complex. The STL was in the process of being developed. Pentiums were state of the art On Fri, 1 Sept 2023, 10:57 Greg Keogh via ozdotnet, <ozdotnet@ozdotnet.com> wrote:
I have never understood the fixation with C++ unless you're in the
business of writing kernels, device drivers, embedded systems, etc.
The library we're phasing out was started around 1997, so you can throw the authors a bone because the world was very different back then. Your main choices were C/C++, VB and Java.
I stand by my opinion that C++ is the most absurdly complex and idiotic language in contemporary use. Ooops! I made a judgement.
*GK*
On Fri, 1 Sept 2023 at 08:44, Greg Keogh via ozdotnet < ozdotnet@ozdotnet.com> wrote:
Folks, it's Friday and I have an anecdote to share before I return to today's coding fiasco. I'll just tell you what happened and try to avoid making judgements, I'll leave that to you.
For about 4 years we've had an Azure hosted Web API/service driving a moderately complex Blazor app and some other smaller clients. The service hosted a C++ library that did the heavy lifting of generating cross-tabulation reports. The trouble was, that the service would randomly crash deep inside the C++ dll and it would leave no useful diagnostic evidence, usually just a hint about some kind of memory access violation. It would never crash in testing, only in Azure. I presume here is a way to diagnose this sort of crash in Azure hosting, but you probably need the minidump and symbol files out of the C++ compile, I'm not exactly sure, as I just couldn't face the toil, and by great luck it was due for replacement anyway. Some very large US companies were suffering from the crash interruptions and there was a serious risk that we could lose their business.
The lucky part is that the huge C++ codebase was already being rewritten in C#, so we went into a frenzy of continued conversion and testing, and the C# replacement is now about 90% rolled-out. and guess what?! ... The random crashes are gone and one customer even sent us a message of thanks for the new reliability.
We did have a few small unhandled exceptions, but I simply went to the Azure portal logs and the stack trace pointed us straight to the problem point. We could usually publish a fix within half an hour.
So years of random C++ crashes were completely cured by a C# rewrite.
*Greg K* -- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
-- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
-- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
Greg next time get a dump and you can use VS to debug it back on your own machine. The AV could be something as simple as de-referencing a null. Merge something like this into the registry https://github.com/dotnet/project-system/blob/main/docs/repo/content/AlwaysS... just swapping for your own process if you can do that on Azure. If you can't merge keys, launch with procdump.exe. And I'm not just saying this because I work on VS, but Visual Studio's dump debugging is very good - nothing better. For managed code it's even more amazing - I use it literally every day to debug dumps. From: mike smith via ozdotnet <ozdotnet@ozdotnet.com> Sent: Friday, September 1, 2023 12:02 PM To: ozDotNet <ozdotnet@ozdotnet.com> Cc: mike smith <meski.oz@gmail.com> Subject: Re: A real C++ vs C# story Back then C++ wasn't so complex. The STL was in the process of being developed. Pentiums were state of the art On Fri, 1 Sept 2023, 10:57 Greg Keogh via ozdotnet, <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> wrote: I have never understood the fixation with C++ unless you're in the business of writing kernels, device drivers, embedded systems, etc. The library we're phasing out was started around 1997, so you can throw the authors a bone because the world was very different back then. Your main choices were C/C++, VB and Java. I stand by my opinion that C++ is the most absurdly complex and idiotic language in contemporary use. Ooops! I made a judgement. GK On Fri, 1 Sept 2023 at 08:44, Greg Keogh via ozdotnet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> wrote: Folks, it's Friday and I have an anecdote to share before I return to today's coding fiasco. I'll just tell you what happened and try to avoid making judgements, I'll leave that to you. For about 4 years we've had an Azure hosted Web API/service driving a moderately complex Blazor app and some other smaller clients. The service hosted a C++ library that did the heavy lifting of generating cross-tabulation reports. The trouble was, that the service would randomly crash deep inside the C++ dll and it would leave no useful diagnostic evidence, usually just a hint about some kind of memory access violation. It would never crash in testing, only in Azure. I presume here is a way to diagnose this sort of crash in Azure hosting, but you probably need the minidump and symbol files out of the C++ compile, I'm not exactly sure, as I just couldn't face the toil, and by great luck it was due for replacement anyway. Some very large US companies were suffering from the crash interruptions and there was a serious risk that we could lose their business. The lucky part is that the huge C++ codebase was already being rewritten in C#, so we went into a frenzy of continued conversion and testing, and the C# replacement is now about 90% rolled-out. and guess what?! ... The random crashes are gone and one customer even sent us a message of thanks for the new reliability. We did have a few small unhandled exceptions, but I simply went to the Azure portal logs and the stack trace pointed us straight to the problem point. We could usually publish a fix within half an hour. So years of random C++ crashes were completely cured by a C# rewrite. Greg K -- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/ -- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/ -- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
You really need to be generating the pdb files and storing them in a symbol server. If you want to view the source code associated with it, store that as well. For every build you want to be able to debug. I used to store it for all builds, and clean up later, because internal builds sometimes crashed in QA. If you don't.... you'd better get good at reading x64 mnemonics... 😜 Mike On Fri, 1 Sept 2023, 08:15 Greg Keogh via ozdotnet, <ozdotnet@ozdotnet.com> wrote:
Folks, it's Friday and I have an anecdote to share before I return to today's coding fiasco. I'll just tell you what happened and try to avoid making judgements, I'll leave that to you.
For about 4 years we've had an Azure hosted Web API/service driving a moderately complex Blazor app and some other smaller clients. The service hosted a C++ library that did the heavy lifting of generating cross-tabulation reports. The trouble was, that the service would randomly crash deep inside the C++ dll and it would leave no useful diagnostic evidence, usually just a hint about some kind of memory access violation. It would never crash in testing, only in Azure. I presume here is a way to diagnose this sort of crash in Azure hosting, but you probably need the minidump and symbol files out of the C++ compile, I'm not exactly sure, as I just couldn't face the toil, and by great luck it was due for replacement anyway. Some very large US companies were suffering from the crash interruptions and there was a serious risk that we could lose their business.
The lucky part is that the huge C++ codebase was already being rewritten in C#, so we went into a frenzy of continued conversion and testing, and the C# replacement is now about 90% rolled-out. and guess what?! ... The random crashes are gone and one customer even sent us a message of thanks for the new reliability.
We did have a few small unhandled exceptions, but I simply went to the Azure portal logs and the stack trace pointed us straight to the problem point. We could usually publish a fix within half an hour.
So years of random C++ crashes were completely cured by a C# rewrite.
*Greg K* -- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
participants (4)
-
David Connors -
David Kean -
Greg Keogh -
mike smith