Worse, in the baseball umpiring exam I mentioned, someone said to me “isn’t that what you’d get if you just asked a whole lot of fans about the rules rather than asking umpires?” There’s probably something important about that. How does it know which of the material it was trained on is valid? Regards, Greg Dr Greg Low 1300SQLSQL (1300 775 775) office | +61 419201410 mobile SQL Down Under | Web: https://sqldownunder.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__sqldownunder.com_&d=DwMFAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=2rgtwrXggQFZiZbisdwDooYFalucb-vLhjG0McaanBZKn0UVuognuHqfHnjp2AVc&m=I23jyX4AKIv9q2x7A3CQAer9PGCjq8R6DwW7BE1IAhZ1JbigKMrMPRCjs6AqW7h3&s=o3oFliHztOF8D9Nbqaa7KQdqC-zkQNXWl4IqnEG58Wc&e=> | About Greg: https://about.me/greg.low<https://urldefense.proofpoint.com/v2/url?u=https-3A__about.me_greg.low&d=DwMFAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=2rgtwrXggQFZiZbisdwDooYFalucb-vLhjG0McaanBZKn0UVuognuHqfHnjp2AVc&m=I23jyX4AKIv9q2x7A3CQAer9PGCjq8R6DwW7BE1IAhZ1JbigKMrMPRCjs6AqW7h3&s=NsAibgiqfCxsyc8m2DBKogKQcs3OqE3mkyCjmpoYxTk&e=> From: mike smith via ozdotnet <ozdotnet@ozdotnet.com> Sent: Friday, February 23, 2024 12:19 PM To: ozDotNet <ozdotnet@ozdotnet.com> Cc: mike smith <meski.oz@gmail.com> Subject: Re: AI "old system views" That makes me wonder if it has any way of differentiating between something it found from a decade ago to more recent data. Mike On Fri, 23 Feb 2024, 11:43 Dr Greg Low via ozdotnet, <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> wrote: Hi Tom, For me, it depends what you want it to do. It certainly can appear to help someone who’s new to an area. For most code writing, I’ve been pretty underwhelmed. As an example, if I ask it to write SQL, I get a very poor outcome. It will use old deprecated views instead of the current system views (that have been around for a decade), and often does things in a convoluted way. What I have been impressed with, is how it can help you understand acronyms, etc. Quite amazing. I’ve also been pretty impressed with using it go generate some test data, including in multiple languages. And the test data is fairly believable. If I ask it for family names, and I also ask for Chinese, it does pick common Chinese family names in the test output. That’s pretty impressive. It can do a reasonable job of things like “here’s some DAX code, can you simplify it?” It often can. Or “here’s a regular expression, can you explain what it does?” and it does that just fine. I’ve seen people happily using it to explain code that they don’t understand, or to (sort of) document some code. But it also is so confident on things, yet so wrong. I gave it a 25 question baseball umpire test the other day. It was 100% confident sounding, but 40% correct. The weird thing is that some of the questions that it got right, are things that new human umpires often get wrong. Yet for simpler questions, it would say that something legal is illegal. It’s certainly interesting, but it’s very much a work in progress. It will be part of our futures. Regards, Greg Dr Greg Low 1300SQLSQL (1300 775 775) office | +61 419201410 mobile SQL Down Under | Web: https://sqldownunder.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__sqldownunder.com_&d=DwMFAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=2rgtwrXggQFZiZbisdwDooYFalucb-vLhjG0McaanBZKn0UVuognuHqfHnjp2AVc&m=I23jyX4AKIv9q2x7A3CQAer9PGCjq8R6DwW7BE1IAhZ1JbigKMrMPRCjs6AqW7h3&s=o3oFliHztOF8D9Nbqaa7KQdqC-zkQNXWl4IqnEG58Wc&e=> | About Greg: https://about.me/greg.low<https://urldefense.proofpoint.com/v2/url?u=https-3A__about.me_greg.low&d=DwMFAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=2rgtwrXggQFZiZbisdwDooYFalucb-vLhjG0McaanBZKn0UVuognuHqfHnjp2AVc&m=I23jyX4AKIv9q2x7A3CQAer9PGCjq8R6DwW7BE1IAhZ1JbigKMrMPRCjs6AqW7h3&s=NsAibgiqfCxsyc8m2DBKogKQcs3OqE3mkyCjmpoYxTk&e=> From: Tom Gao via ozdotnet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> Sent: Friday, February 23, 2024 11:58 AM To: ozDotNet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> Cc: Tom Gao <tom@tomgao.com<mailto:tom@tomgao.com>> Subject: AI Hi guys, I haven't posted in a few years and haven't been on the tools for a long time now as well. I'm on a panel on a digital conference coming up in march. We had a pre meeting today and the topic of AI came up. Two of the panelist said cited CBA and Westpac using AI and were able to save 30% on development effort. Personally I just finished an AI course my view is quite the opposite. My personal opinion of the generative AI space and AI in general having spent time with the academics is that the benefits are significantly over inflated. I want to get some other opinions if you are seeing any significant benefit and that I may be just out of touch or not aware. Thanks, Tom -- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/

Here's the article. https://www.afr.com/technology/commbank-claims-ai-is-already-making-it-work-... So what does everyone think about the claim that 30% efficiency? ie to me as head of IT instead of hiring 10 developers to build applications now only 7 is required. To me that just doesn't seem realistic. On Fri, Feb 23, 2024 at 3:04 PM Tom Rutter via ozdotnet < ozdotnet@ozdotnet.com> wrote:

On Fri, 23 Feb 2024 at 12:32, Dr Greg Low via ozdotnet < ozdotnet@ozdotnet.com> wrote:

...

Worse, in the baseball umpiring exam I mentioned, someone said to me “isn’t that what you’d get if you just asked a whole lot of fans about the rules rather than asking umpires?”

There’s probably something important about that. How does it know which of the material it was trained on is valid?

Yep that’s the thing, it doesn’t actually know what is “valid”. From what I understand it just gets “pushed” in certain directions during training and not in all cases obviously.

My experience with things like ChatGPT so far basically brings it down to a better Google really. Early days

...

Regards,

Greg

Dr Greg Low

1300SQLSQL (1300 775 775) office | +61 419201410 mobile

SQL Down Under | Web: https://sqldownunder.com <https://urldefense.proofpoint.com/v2/url?u=https-3A__sqldownunder.com_&d=DwM...> | About Greg: https://about.me/greg.low <https://urldefense.proofpoint.com/v2/url?u=https-3A__about.me_greg.low&d=DwM...>

*From:* mike smith via ozdotnet <ozdotnet@ozdotnet.com>

*Sent:* Friday, February 23, 2024 12:19 PM *To:* ozDotNet <ozdotnet@ozdotnet.com> *Cc:* mike smith <meski.oz@gmail.com> *Subject:* Re: AI

"old system views"

That makes me wonder if it has any way of differentiating between something it found from a decade ago to more recent data.

Mike

On Fri, 23 Feb 2024, 11:43 Dr Greg Low via ozdotnet, < ozdotnet@ozdotnet.com> wrote:

Hi Tom,

For me, it depends what you want it to do. It certainly can appear to help someone who’s new to an area.

For most code writing, I’ve been pretty underwhelmed. As an example, if I ask it to write SQL, I get a very poor outcome. It will use old deprecated views instead of the current system views (that have been around for a decade), and often does things in a convoluted way.

What I have been impressed with, is how it can help you understand acronyms, etc. Quite amazing. I’ve also been pretty impressed with using it go generate some test data, including in multiple languages. And the test data is fairly believable. If I ask it for family names, and I also ask for Chinese, it does pick common Chinese family names in the test output. That’s pretty impressive.

It can do a reasonable job of things like “here’s some DAX code, can you simplify it?” It often can. Or “here’s a regular expression, can you explain what it does?” and it does that just fine. I’ve seen people happily using it to explain code that they don’t understand, or to (sort of) document some code.

But it also is so confident on things, yet so wrong. I gave it a 25 question baseball umpire test the other day. It was 100% confident sounding, but 40% correct. The weird thing is that some of the questions that it got right, are things that new human umpires often get wrong. Yet for simpler questions, it would say that something legal is illegal.

It’s certainly interesting, but it’s very much a work in progress. It will be part of our futures.

Regards,

Greg

Dr Greg Low

1300SQLSQL (1300 775 775) office | +61 419201410 mobile

SQL Down Under | Web: https://sqldownunder.com <https://urldefense.proofpoint.com/v2/url?u=https-3A__sqldownunder.com_&d=DwM...> | About Greg: https://about.me/greg.low <https://urldefense.proofpoint.com/v2/url?u=https-3A__about.me_greg.low&d=DwM...>

*From:* Tom Gao via ozdotnet <ozdotnet@ozdotnet.com> *Sent:* Friday, February 23, 2024 11:58 AM *To:* ozDotNet <ozdotnet@ozdotnet.com> *Cc:* Tom Gao <tom@tomgao.com> *Subject:* AI

Hi guys, I haven't posted in a few years and haven't been on the tools for a long time now as well. I'm on a panel on a digital conference coming up in march. We had a pre meeting today and the topic of AI came up. Two of the panelist said cited CBA and Westpac using AI and were able to save 30% on development effort.

Personally I just finished an AI course my view is quite the opposite. My personal opinion of the generative AI space and AI in general having spent time with the academics is that the benefits are significantly over inflated.

I want to get some other opinions if you are seeing any significant benefit and that I may be just out of touch or not aware.

Thanks,

Tom

-- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/

-- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/

-- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/

If you’re a seasoned practitioner in the language it’s probably sitting around 1-2%, because it can help with some code conversions. Copilot code completions are amateur. For .net resharper/rider etc is actually a much better option than copilot, practical use of that is probably around 15% productivity increase. Copilot does help if you don’t know the language or technology, which maybe commbank staff are not skilled up enough? ;) On Sat, 24 Feb 2024 at 6:27 pm, DotNet Dude via ozdotnet < ozdotnet@ozdotnet.com> wrote:

I’m not a subscriber to read the entire article but 30% is a load of BS at this stage

On Sat, 24 Feb 2024 at 15:18, Tom Gao via ozdotnet <ozdotnet@ozdotnet.com> wrote:

...

Here's the article.

https://www.afr.com/technology/commbank-claims-ai-is-already-making-it-work-...

So what does everyone think about the claim that 30% efficiency? ie to me as head of IT instead of hiring 10 developers to build applications now only 7 is required. To me that just doesn't seem realistic.

On Fri, Feb 23, 2024 at 3:04 PM Tom Rutter via ozdotnet < ozdotnet@ozdotnet.com> wrote:

...

On Fri, 23 Feb 2024 at 12:32, Dr Greg Low via ozdotnet < ozdotnet@ozdotnet.com> wrote:

...

Worse, in the baseball umpiring exam I mentioned, someone said to me “isn’t that what you’d get if you just asked a whole lot of fans about the rules rather than asking umpires?”

There’s probably something important about that. How does it know which of the material it was trained on is valid?

Yep that’s the thing, it doesn’t actually know what is “valid”. From what I understand it just gets “pushed” in certain directions during training and not in all cases obviously.

My experience with things like ChatGPT so far basically brings it down to a better Google really. Early days

...

Regards,

Greg

Dr Greg Low

1300SQLSQL (1300 775 775) office | +61 419201410 mobile

SQL Down Under | Web: https://sqldownunder.com <https://urldefense.proofpoint.com/v2/url?u=https-3A__sqldownunder.com_&d=DwM...> | About Greg: https://about.me/greg.low <https://urldefense.proofpoint.com/v2/url?u=https-3A__about.me_greg.low&d=DwM...>

*From:* mike smith via ozdotnet <ozdotnet@ozdotnet.com>

*Sent:* Friday, February 23, 2024 12:19 PM *To:* ozDotNet <ozdotnet@ozdotnet.com> *Cc:* mike smith <meski.oz@gmail.com> *Subject:* Re: AI

"old system views"

That makes me wonder if it has any way of differentiating between something it found from a decade ago to more recent data.

Mike

On Fri, 23 Feb 2024, 11:43 Dr Greg Low via ozdotnet, < ozdotnet@ozdotnet.com> wrote:

Hi Tom,

For me, it depends what you want it to do. It certainly can appear to help someone who’s new to an area.

For most code writing, I’ve been pretty underwhelmed. As an example, if I ask it to write SQL, I get a very poor outcome. It will use old deprecated views instead of the current system views (that have been around for a decade), and often does things in a convoluted way.

What I have been impressed with, is how it can help you understand acronyms, etc. Quite amazing. I’ve also been pretty impressed with using it go generate some test data, including in multiple languages. And the test data is fairly believable. If I ask it for family names, and I also ask for Chinese, it does pick common Chinese family names in the test output. That’s pretty impressive.

It can do a reasonable job of things like “here’s some DAX code, can you simplify it?” It often can. Or “here’s a regular expression, can you explain what it does?” and it does that just fine. I’ve seen people happily using it to explain code that they don’t understand, or to (sort of) document some code.

But it also is so confident on things, yet so wrong. I gave it a 25 question baseball umpire test the other day. It was 100% confident sounding, but 40% correct. The weird thing is that some of the questions that it got right, are things that new human umpires often get wrong. Yet for simpler questions, it would say that something legal is illegal.

It’s certainly interesting, but it’s very much a work in progress. It will be part of our futures.

Regards,

Greg

Dr Greg Low

1300SQLSQL (1300 775 775) office | +61 419201410 mobile

SQL Down Under | Web: https://sqldownunder.com <https://urldefense.proofpoint.com/v2/url?u=https-3A__sqldownunder.com_&d=DwM...> | About Greg: https://about.me/greg.low <https://urldefense.proofpoint.com/v2/url?u=https-3A__about.me_greg.low&d=DwM...>

*From:* Tom Gao via ozdotnet <ozdotnet@ozdotnet.com> *Sent:* Friday, February 23, 2024 11:58 AM *To:* ozDotNet <ozdotnet@ozdotnet.com> *Cc:* Tom Gao <tom@tomgao.com> *Subject:* AI

Hi guys, I haven't posted in a few years and haven't been on the tools for a long time now as well. I'm on a panel on a digital conference coming up in march. We had a pre meeting today and the topic of AI came up. Two of the panelist said cited CBA and Westpac using AI and were able to save 30% on development effort.

Personally I just finished an AI course my view is quite the opposite. My personal opinion of the generative AI space and AI in general having spent time with the academics is that the benefits are significantly over inflated.

I want to get some other opinions if you are seeing any significant benefit and that I may be just out of touch or not aware.

Thanks,

Tom

-- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/

-- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/

-- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/

-- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/

-- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/

On Sun, 28 Jul 2024 at 23:16, Tom Gao via ozdotnet <ozdotnet@ozdotnet.com> wrote:

I wanted to get some feedback from you guys last time michael said maybe about 1-2% in improvement in usefulness to a seasoned dev. I've had ongoing robust conversations. Because I feel the benefits are completely overstated in general (open to being corrected).

I don't think the benefit is overrated and we see huge value in the use of AI across all of the dev work we do. I think you're possibly looking at it glass half empty instead of glass half full. The question is not whether the AI is smarter than you, can write better code than you - the question is whether you get a great productivity boost in day to day activities. Co-pilot smashes boring drudge work like making constructors out of the park. You can use ChatGPT to break the back of time consuming but easy work - for example, upload a CSV and then ask it to write you the DDL for a table to store that and a stored procedure to do CRUD operations etc. It churns it out instantly. If you asked me to write an UPSERT statement in T-SQL or create a user in MySQL I would have to Google it. They're tasks that are easy, but also tasks that I seldom do so don't have the syntax front of mind. I can just leave out the middle-man and get the AI to do it for me.

The most senior tech lead in my team believe it's a negative benefit. As in it can't understand complex business analyst requirements, nor debug codes, nor find / correct defects.

All of our PRs are reviewed by AI before humans touch them (via Code Rabbit). People reviewing PRs are generally more senior, more expensive and time poor. Code Rabbit produces *really* good summaries of what is in the PR and the implications. It provides a lot of low level commentary that is useful to the dev's themselves before asking for a review. You can talk back and forth with Code Rabbit in the comments under the PR in GitHub and ask for suggestions etc.

The effort to prompt engineer and also to correct what's generated is also understated.

What sort of things are you asking it to do? If you're trying to get an LLM to write conceptually complex things then that might be where you have friction. Here's an example of a CodeRabbit review for a simple change done today in one of our systems together with example of dev conversation. The reviews of complex changes (I could not find a good example without internal information) are astonishingly good and the back and forth with the bot can be very nuanced (arguing the point on why there isn't actually a race condition in some code because it isn't considering this or that). This level of review / suggestion / fix cycle happens before a senior considers the PR. The poems are a nice touch :) [image: image.png] Conversation on that change with CodeRabbit: [image: image.png]

Doesn't surprise me. We recently bought a new house and I was trying to make sense of all of the town plan documents during the settlement period. I just uploaded it all to ChatGPT and asked for an explanation of everything including impacts of easements etc. I sent the chat to my town planner and he was pretty shocked at how good it was. It is not going to delete professions but unless you're onboard with it you won't be able to meet the performance expectations of your customers / employers. On Tue, 30 Jul 2024 at 16:55, Dr Greg Low <greg@sqldownunder.com> wrote:

Was at an eye specialist this morning for a check-up.

When we were finished, he wanted to talk about AI.

He showed me a system they've started using. It combines input from around six separate technical systems that they use, reads all of the patient history, clinical notes, and compares the captured images across all the years I've been to see him.

As he writes his clinical notes in for the day, it wrote a summary of unbelievably high quality. He told me how it was way better than anything he could ever do, particularly the way it reviewed everything from the past and compared it (in detail) to the current.

He was genuinely concerned about what sort of job he's going to have in future.

It's not replacing him yet, but it's completely changed his ability to work at a much higher level.

Regards,

Greg

Dr Greg Low

1300SQLSQL (1300 775 775) office | +61 419201410 mobile

SQL Down Under | Web: https://sqldownunder.com <https://urldefense.proofpoint.com/v2/url?u=https-3A__sqldownunder.com_&d=DwM...> | About Greg: https://about.me/greg.low <https://urldefense.proofpoint.com/v2/url?u=https-3A__about.me_greg.low&d=DwM...>

*From:* David Connors via ozdotnet <ozdotnet@ozdotnet.com> *Sent:* Tuesday, July 30, 2024 2:17 PM *To:* ozDotNet <ozdotnet@ozdotnet.com> *Cc:* David Connors <david@connors.com> *Subject:* Re: AI

On Sun, 28 Jul 2024 at 23:16, Tom Gao via ozdotnet <ozdotnet@ozdotnet.com> wrote:

I wanted to get some feedback from you guys last time michael said maybe about 1-2% in improvement in usefulness to a seasoned dev. I've had ongoing robust conversations. Because I feel the benefits are completely overstated in general (open to being corrected).

I don't think the benefit is overrated and we see huge value in the use of AI across all of the dev work we do. I think you're possibly looking at it glass half empty instead of glass half full. The question is not whether the AI is smarter than you, can write better code than you - the question is whether you get a great productivity boost in day to day activities.

Co-pilot smashes boring drudge work like making constructors out of the park. You can use ChatGPT to break the back of time consuming but easy work - for example, upload a CSV and then ask it to write you the DDL for a table to store that and a stored procedure to do CRUD operations etc. It churns it out instantly.

If you asked me to write an UPSERT statement in T-SQL or create a user in MySQL I would have to Google it. They're tasks that are easy, but also tasks that I seldom do so don't have the syntax front of mind. I can just leave out the middle-man and get the AI to do it for me.

The most senior tech lead in my team believe it's a negative benefit. As in it can't understand complex business analyst requirements, nor debug codes, nor find / correct defects.

All of our PRs are reviewed by AI before humans touch them (via Code Rabbit). People reviewing PRs are generally more senior, more expensive and time poor. Code Rabbit produces *really* good summaries of what is in the PR and the implications. It provides a lot of low level commentary that is useful to the dev's themselves before asking for a review. You can talk back and forth with Code Rabbit in the comments under the PR in GitHub and ask for suggestions etc.

The effort to prompt engineer and also to correct what's generated is also understated.

What sort of things are you asking it to do? If you're trying to get an LLM to write conceptually complex things then that might be where you have friction.

Here's an example of a CodeRabbit review for a simple change done today in one of our systems together with example of dev conversation. The reviews of complex changes (I could not find a good example without internal information) are astonishingly good and the back and forth with the bot can be very nuanced (arguing the point on why there isn't actually a race condition in some code because it isn't considering this or that). This level of review / suggestion / fix cycle happens before a senior considers the PR.

The poems are a nice touch :)

Conversation on that change with CodeRabbit:

They'll be fine. Medicine in a borderline impenetrable stitch up anyway. End game for your ophthalmologist is a few hundred grand a year box ticking AI output and nodding thoughtfully. The universities will resist this for years and year to come. One of our guys still has a bit to go on his degree and his professors tut tut the use of AI to the students. 🙄 Ignore it at your peril. LLMs and associated tech are going to put a LOT of people out of work - you need to make sure you're the one putting people out of work - not being put out of work. The job of a good software engineer is safe for a long time. The performance expectations will increase though. On Tue, 30 Jul 2024 at 18:39, Dr Greg Low <greg@sqldownunder.com> wrote:

I should add that the reason he raised it was being literally scared about the future. And not just for himself, the future of work is suddenly becoming apparent to him.

Regards

Greg

Dr Greg Low SQL Down Under Pty Ltd +61 419201410

------------------------------ *From:* David Connors via ozdotnet <ozdotnet@ozdotnet.com> *Sent:* Tuesday, July 30, 2024 5:59:37 PM

*To:* ozDotNet <ozdotnet@ozdotnet.com> *Cc:* David Connors <david@connors.com> *Subject:* Re: AI

Doesn't surprise me. We recently bought a new house and I was trying to make sense of all of the town plan documents during the settlement period. I just uploaded it all to ChatGPT and asked for an explanation of everything including impacts of easements etc.

I sent the chat to my town planner and he was pretty shocked at how good it was.

It is not going to delete professions but unless you're onboard with it you won't be able to meet the performance expectations of your customers / employers.

On Tue, 30 Jul 2024 at 16:55, Dr Greg Low <greg@sqldownunder.com> wrote:

Was at an eye specialist this morning for a check-up.

When we were finished, he wanted to talk about AI.

He showed me a system they've started using. It combines input from around six separate technical systems that they use, reads all of the patient history, clinical notes, and compares the captured images across all the years I've been to see him.

As he writes his clinical notes in for the day, it wrote a summary of unbelievably high quality. He told me how it was way better than anything he could ever do, particularly the way it reviewed everything from the past and compared it (in detail) to the current.

He was genuinely concerned about what sort of job he's going to have in future.

It's not replacing him yet, but it's completely changed his ability to work at a much higher level.

Regards,

Greg

Dr Greg Low

1300SQLSQL (1300 775 775) office | +61 419201410 mobile

SQL Down Under | Web: https://sqldownunder.com <https://urldefense.proofpoint.com/v2/url?u=https-3A__sqldownunder.com_&d=DwM...> | About Greg: https://about.me/greg.low <https://urldefense.proofpoint.com/v2/url?u=https-3A__about.me_greg.low&d=DwM...>

*From:* David Connors via ozdotnet <ozdotnet@ozdotnet.com> *Sent:* Tuesday, July 30, 2024 2:17 PM *To:* ozDotNet <ozdotnet@ozdotnet.com> *Cc:* David Connors <david@connors.com> *Subject:* Re: AI

On Sun, 28 Jul 2024 at 23:16, Tom Gao via ozdotnet <ozdotnet@ozdotnet.com> wrote:

I wanted to get some feedback from you guys last time michael said maybe about 1-2% in improvement in usefulness to a seasoned dev. I've had ongoing robust conversations. Because I feel the benefits are completely overstated in general (open to being corrected).

I don't think the benefit is overrated and we see huge value in the use of AI across all of the dev work we do. I think you're possibly looking at it glass half empty instead of glass half full. The question is not whether the AI is smarter than you, can write better code than you - the question is whether you get a great productivity boost in day to day activities.

Co-pilot smashes boring drudge work like making constructors out of the park. You can use ChatGPT to break the back of time consuming but easy work - for example, upload a CSV and then ask it to write you the DDL for a table to store that and a stored procedure to do CRUD operations etc. It churns it out instantly.

If you asked me to write an UPSERT statement in T-SQL or create a user in MySQL I would have to Google it. They're tasks that are easy, but also tasks that I seldom do so don't have the syntax front of mind. I can just leave out the middle-man and get the AI to do it for me.

The most senior tech lead in my team believe it's a negative benefit. As in it can't understand complex business analyst requirements, nor debug codes, nor find / correct defects.

All of our PRs are reviewed by AI before humans touch them (via Code Rabbit). People reviewing PRs are generally more senior, more expensive and time poor. Code Rabbit produces *really* good summaries of what is in the PR and the implications. It provides a lot of low level commentary that is useful to the dev's themselves before asking for a review. You can talk back and forth with Code Rabbit in the comments under the PR in GitHub and ask for suggestions etc.

The effort to prompt engineer and also to correct what's generated is also understated.

What sort of things are you asking it to do? If you're trying to get an LLM to write conceptually complex things then that might be where you have friction.

Here's an example of a CodeRabbit review for a simple change done today in one of our systems together with example of dev conversation. The reviews of complex changes (I could not find a good example without internal information) are astonishingly good and the back and forth with the bot can be very nuanced (arguing the point on why there isn't actually a race condition in some code because it isn't considering this or that). This level of review / suggestion / fix cycle happens before a senior considers the PR.

The poems are a nice touch :)

Conversation on that change with CodeRabbit:

No old friend at Blackburn Dr Greg Low SQL Down Under Pty Ltd +61 419201410 ________________________________ From: Tom Gao <tom@tomgao.com> Sent: Tuesday, July 30, 2024 6:54:08 PM To: ozDotNet <ozdotnet@ozdotnet.com> Cc: David Connors <david@connors.com>; Dr Greg Low <greg@sqldownunder.com> Subject: Re: AI Hi Greg, funny that I was also at my ophthalmologist this morning there's not many around were you around randwick by any chance ? lol On Tue, Jul 30, 2024 at 6:40 PM Dr Greg Low via ozdotnet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> wrote: I should add that the reason he raised it was being literally scared about the future. And not just for himself, the future of work is suddenly becoming apparent to him. Regards Greg Dr Greg Low SQL Down Under Pty Ltd +61 419201410 ________________________________ From: David Connors via ozdotnet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> Sent: Tuesday, July 30, 2024 5:59:37 PM To: ozDotNet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> Cc: David Connors <david@connors.com<mailto:david@connors.com>> Subject: Re: AI Doesn't surprise me. We recently bought a new house and I was trying to make sense of all of the town plan documents during the settlement period. I just uploaded it all to ChatGPT and asked for an explanation of everything including impacts of easements etc. I sent the chat to my town planner and he was pretty shocked at how good it was. It is not going to delete professions but unless you're onboard with it you won't be able to meet the performance expectations of your customers / employers. On Tue, 30 Jul 2024 at 16:55, Dr Greg Low <greg@sqldownunder.com<mailto:greg@sqldownunder.com>> wrote: Was at an eye specialist this morning for a check-up. When we were finished, he wanted to talk about AI. He showed me a system they've started using. It combines input from around six separate technical systems that they use, reads all of the patient history, clinical notes, and compares the captured images across all the years I've been to see him. As he writes his clinical notes in for the day, it wrote a summary of unbelievably high quality. He told me how it was way better than anything he could ever do, particularly the way it reviewed everything from the past and compared it (in detail) to the current. He was genuinely concerned about what sort of job he's going to have in future. It's not replacing him yet, but it's completely changed his ability to work at a much higher level. Regards, Greg Dr Greg Low 1300SQLSQL (1300 775 775) office | +61 419201410 mobile SQL Down Under | Web: https://sqldownunder.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__sqldownunder.com_&d=DwMFAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=2rgtwrXggQFZiZbisdwDooYFalucb-vLhjG0McaanBZKn0UVuognuHqfHnjp2AVc&m=I23jyX4AKIv9q2x7A3CQAer9PGCjq8R6DwW7BE1IAhZ1JbigKMrMPRCjs6AqW7h3&s=o3oFliHztOF8D9Nbqaa7KQdqC-zkQNXWl4IqnEG58Wc&e=> | About Greg: https://about.me/greg.low<https://urldefense.proofpoint.com/v2/url?u=https-3A__about.me_greg.low&d=DwMFAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=2rgtwrXggQFZiZbisdwDooYFalucb-vLhjG0McaanBZKn0UVuognuHqfHnjp2AVc&m=I23jyX4AKIv9q2x7A3CQAer9PGCjq8R6DwW7BE1IAhZ1JbigKMrMPRCjs6AqW7h3&s=NsAibgiqfCxsyc8m2DBKogKQcs3OqE3mkyCjmpoYxTk&e=> From: David Connors via ozdotnet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> Sent: Tuesday, July 30, 2024 2:17 PM To: ozDotNet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> Cc: David Connors <david@connors.com<mailto:david@connors.com>> Subject: Re: AI On Sun, 28 Jul 2024 at 23:16, Tom Gao via ozdotnet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> wrote: I wanted to get some feedback from you guys last time michael said maybe about 1-2% in improvement in usefulness to a seasoned dev. I've had ongoing robust conversations. Because I feel the benefits are completely overstated in general (open to being corrected). I don't think the benefit is overrated and we see huge value in the use of AI across all of the dev work we do. I think you're possibly looking at it glass half empty instead of glass half full. The question is not whether the AI is smarter than you, can write better code than you - the question is whether you get a great productivity boost in day to day activities. Co-pilot smashes boring drudge work like making constructors out of the park. You can use ChatGPT to break the back of time consuming but easy work - for example, upload a CSV and then ask it to write you the DDL for a table to store that and a stored procedure to do CRUD operations etc. It churns it out instantly. If you asked me to write an UPSERT statement in T-SQL or create a user in MySQL I would have to Google it. They're tasks that are easy, but also tasks that I seldom do so don't have the syntax front of mind. I can just leave out the middle-man and get the AI to do it for me. The most senior tech lead in my team believe it's a negative benefit. As in it can't understand complex business analyst requirements, nor debug codes, nor find / correct defects. All of our PRs are reviewed by AI before humans touch them (via Code Rabbit). People reviewing PRs are generally more senior, more expensive and time poor. Code Rabbit produces *really* good summaries of what is in the PR and the implications. It provides a lot of low level commentary that is useful to the dev's themselves before asking for a review. You can talk back and forth with Code Rabbit in the comments under the PR in GitHub and ask for suggestions etc. The effort to prompt engineer and also to correct what's generated is also understated. What sort of things are you asking it to do? If you're trying to get an LLM to write conceptually complex things then that might be where you have friction. Here's an example of a CodeRabbit review for a simple change done today in one of our systems together with example of dev conversation. The reviews of complex changes (I could not find a good example without internal information) are astonishingly good and the back and forth with the bot can be very nuanced (arguing the point on why there isn't actually a race condition in some code because it isn't considering this or that). This level of review / suggestion / fix cycle happens before a senior considers the PR. The poems are a nice touch :) [cid:ii_19102d564454cff312] Conversation on that change with CodeRabbit: [cid:ii_19102d564445b16b21] -- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/

There has been resistance to continuing to train radiologists in the same numbers. "In a 2023 study, authors reported a 99.1 sensitivity rate for AI on abnormal radiographs in comparison to a 72.3 percent sensitivity for radiologist reports. Autonomous AI also yielded a 6.3 percent higher sensitivity than reporting radiologists for critical abnormal X-rays." There are many tasks like this that AI systems now routinely outperform humans. I was impressed with dome new small, limited purpose robots, like a relatively new one that takes blood. It is placed on your arm, and it looks through your arm, and does a 3D analysis of your veins, venal flow, and damage. It then goes directly for the best spot to get blood. Humans just can't do these things. What it does is vastly reduce the skill level required and increase availability of such services. Regards Greg Dr Greg Low SQL Down Under Pty Ltd +61 419201410 ________________________________ From: Scott <scott.barnes@gmail.com> Sent: Tuesday, July 30, 2024 8:37:40 PM To: ozDotNet <ozdotnet@ozdotnet.com> Cc: Tom Gao <tom@tomgao.com>; David Connors <david@connors.com>; Dr Greg Low <greg@sqldownunder.com> Subject: Re: AI omfg. i forgot about this list :D It may replace Business Analyst and partially Architects one day. I'm currently dealing with a constant battle between Developers and Business Analysts over daily ownership of requirements that address solutioning issues. The circuit breaker has been using the entire Jira tickets for a project, feeding them to AI Crew Agents (https://github.com/crewAIInc/crewAI), and having them work overnight on creating an end-to-end technical specifications document based on the existing code base (legacy) combined with the proposed user stories. I also use some TOGAF templates to test a concise formatting option. The results are remarkably professional and accurate compared to the human-centric approach used as a baseline comparison. When multiple AI agents share a context and each has a unique role, it opens up different approaches to communication issues or problem-solving in terms of legacy/reverse engineering mixed with future technical solutioning. It still often hallucinates in critical areas of thought, but i'd say it accelerates the work required here by about 60-70% at the very least (much like it does with C++ code in Unreal Engine, it gives you the breadcrumb trail at the very least to go shape towards a direction you need to head but won't make the game for you sort of thing). Another practical use case at the moment is in the travel industry. Often, the GDS systems have an open text field for additional but pertinent information for accommodations (e.g., key is under a plant, towels need to be washed, etc.), and there is no uniform standard for storing that information (because the travel industry is rarely ever consistent). Using AI, we can feed it a series of specific templates and ask it to match the criteria based on the types of remarks (think of it as a cascade of chaining different types). I don't think it's currently anywhere near a stage where it's a job replacement thing, but it's definitely getting there and shaping data towards "source of truth" practical usage. On Tue, Jul 30, 2024 at 7:26 PM Dr Greg Low via ozdotnet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> wrote: No old friend at Blackburn Dr Greg Low SQL Down Under Pty Ltd +61 419201410 ________________________________ From: Tom Gao <tom@tomgao.com<mailto:tom@tomgao.com>> Sent: Tuesday, July 30, 2024 6:54:08 PM To: ozDotNet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> Cc: David Connors <david@connors.com<mailto:david@connors.com>>; Dr Greg Low <greg@sqldownunder.com<mailto:greg@sqldownunder.com>> Subject: Re: AI Hi Greg, funny that I was also at my ophthalmologist this morning there's not many around were you around randwick by any chance ? lol On Tue, Jul 30, 2024 at 6:40 PM Dr Greg Low via ozdotnet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> wrote: I should add that the reason he raised it was being literally scared about the future. And not just for himself, the future of work is suddenly becoming apparent to him. Regards Greg Dr Greg Low SQL Down Under Pty Ltd +61 419201410 ________________________________ From: David Connors via ozdotnet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> Sent: Tuesday, July 30, 2024 5:59:37 PM To: ozDotNet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> Cc: David Connors <david@connors.com<mailto:david@connors.com>> Subject: Re: AI Doesn't surprise me. We recently bought a new house and I was trying to make sense of all of the town plan documents during the settlement period. I just uploaded it all to ChatGPT and asked for an explanation of everything including impacts of easements etc. I sent the chat to my town planner and he was pretty shocked at how good it was. It is not going to delete professions but unless you're onboard with it you won't be able to meet the performance expectations of your customers / employers. On Tue, 30 Jul 2024 at 16:55, Dr Greg Low <greg@sqldownunder.com<mailto:greg@sqldownunder.com>> wrote: Was at an eye specialist this morning for a check-up. When we were finished, he wanted to talk about AI. He showed me a system they've started using. It combines input from around six separate technical systems that they use, reads all of the patient history, clinical notes, and compares the captured images across all the years I've been to see him. As he writes his clinical notes in for the day, it wrote a summary of unbelievably high quality. He told me how it was way better than anything he could ever do, particularly the way it reviewed everything from the past and compared it (in detail) to the current. He was genuinely concerned about what sort of job he's going to have in future. It's not replacing him yet, but it's completely changed his ability to work at a much higher level. Regards, Greg Dr Greg Low 1300SQLSQL (1300 775 775) office | +61 419201410 mobile SQL Down Under | Web: https://sqldownunder.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__sqldownunder.com_&d=DwMFAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=2rgtwrXggQFZiZbisdwDooYFalucb-vLhjG0McaanBZKn0UVuognuHqfHnjp2AVc&m=I23jyX4AKIv9q2x7A3CQAer9PGCjq8R6DwW7BE1IAhZ1JbigKMrMPRCjs6AqW7h3&s=o3oFliHztOF8D9Nbqaa7KQdqC-zkQNXWl4IqnEG58Wc&e=> | About Greg: https://about.me/greg.low<https://urldefense.proofpoint.com/v2/url?u=https-3A__about.me_greg.low&d=DwMFAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=2rgtwrXggQFZiZbisdwDooYFalucb-vLhjG0McaanBZKn0UVuognuHqfHnjp2AVc&m=I23jyX4AKIv9q2x7A3CQAer9PGCjq8R6DwW7BE1IAhZ1JbigKMrMPRCjs6AqW7h3&s=NsAibgiqfCxsyc8m2DBKogKQcs3OqE3mkyCjmpoYxTk&e=> From: David Connors via ozdotnet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> Sent: Tuesday, July 30, 2024 2:17 PM To: ozDotNet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> Cc: David Connors <david@connors.com<mailto:david@connors.com>> Subject: Re: AI On Sun, 28 Jul 2024 at 23:16, Tom Gao via ozdotnet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> wrote: I wanted to get some feedback from you guys last time michael said maybe about 1-2% in improvement in usefulness to a seasoned dev. I've had ongoing robust conversations. Because I feel the benefits are completely overstated in general (open to being corrected). I don't think the benefit is overrated and we see huge value in the use of AI across all of the dev work we do. I think you're possibly looking at it glass half empty instead of glass half full. The question is not whether the AI is smarter than you, can write better code than you - the question is whether you get a great productivity boost in day to day activities. Co-pilot smashes boring drudge work like making constructors out of the park. You can use ChatGPT to break the back of time consuming but easy work - for example, upload a CSV and then ask it to write you the DDL for a table to store that and a stored procedure to do CRUD operations etc. It churns it out instantly. If you asked me to write an UPSERT statement in T-SQL or create a user in MySQL I would have to Google it. They're tasks that are easy, but also tasks that I seldom do so don't have the syntax front of mind. I can just leave out the middle-man and get the AI to do it for me. The most senior tech lead in my team believe it's a negative benefit. As in it can't understand complex business analyst requirements, nor debug codes, nor find / correct defects. All of our PRs are reviewed by AI before humans touch them (via Code Rabbit). People reviewing PRs are generally more senior, more expensive and time poor. Code Rabbit produces *really* good summaries of what is in the PR and the implications. It provides a lot of low level commentary that is useful to the dev's themselves before asking for a review. You can talk back and forth with Code Rabbit in the comments under the PR in GitHub and ask for suggestions etc. The effort to prompt engineer and also to correct what's generated is also understated. What sort of things are you asking it to do? If you're trying to get an LLM to write conceptually complex things then that might be where you have friction. Here's an example of a CodeRabbit review for a simple change done today in one of our systems together with example of dev conversation. The reviews of complex changes (I could not find a good example without internal information) are astonishingly good and the back and forth with the bot can be very nuanced (arguing the point on why there isn't actually a race condition in some code because it isn't considering this or that). This level of review / suggestion / fix cycle happens before a senior considers the PR. The poems are a nice touch :) [cid:ii_191032a48ef4cff312] Conversation on that change with CodeRabbit: [cid:ii_191032a48ef5b16b21] -- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/ -- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/

Indeed, I would not be surprised to hear a lack of any of it locally. Have worked with teams in a variety of hospitals here and it's underwhelming. My brother runs an ER in QLD and it's depressing discussing the state of play with him there too but in some aspects, they seem more developed than what I find here in VIC. But we're far from the worst. Am currently involved in a replatforming project across a health region in another country and am uninspired daily. The eye specialist I mentioned though, was more than just impressed with what his new system was doing, and it was way beyond what he could achieve by himself. Regards Greg Dr Greg Low SQL Down Under Pty Ltd +61 419201410 ________________________________ From: Tom Gao <tom@tomgao.com> Sent: Tuesday, July 30, 2024 10:45:45 PM To: Dr Greg Low <greg@sqldownunder.com> Cc: Scott <scott.barnes@gmail.com>; ozDotNet <ozdotnet@ozdotnet.com>; David Connors <david@connors.com> Subject: Re: AI Hi Greg, the issue is that I see a lot of people citing articles and research papers they have read. We know academics can also be biased and samples skewed. I guess have you actually experienced it as in actively use something and swear by its efficiency. Also from a sponsorship perspective it'll reduce headcount. Not talking about the future or the opportunities but the efficiencies right now. I did the stanford AI course earlier this year, and we went through all of the medical case studies. At the end of the day it still needed a person to carefully review the AI work. So it comes down to the efficiency side and a productivity tool at best. My wife is an emergency specialist and the director of emergency we often talk about it and she hasn't seen anything AI but then IT in sydney local health is such a mess I wouldn't be surprised. I went through this with smartcity initiatives state government threw hundreds of millions of dollars and everyone swore by smart city and all the technologies at the city level and the benefit it'll bring. The moment funding was gone no council could afford to maintain it and years later the benefit was not there. Same with blockchain.. on and on. On Tue, Jul 30, 2024 at 8:56 PM Dr Greg Low <greg@sqldownunder.com<mailto:greg@sqldownunder.com>> wrote: There has been resistance to continuing to train radiologists in the same numbers. "In a 2023 study, authors reported a 99.1 sensitivity rate for AI on abnormal radiographs in comparison to a 72.3 percent sensitivity for radiologist reports. Autonomous AI also yielded a 6.3 percent higher sensitivity than reporting radiologists for critical abnormal X-rays." There are many tasks like this that AI systems now routinely outperform humans. I was impressed with dome new small, limited purpose robots, like a relatively new one that takes blood. It is placed on your arm, and it looks through your arm, and does a 3D analysis of your veins, venal flow, and damage. It then goes directly for the best spot to get blood. Humans just can't do these things. What it does is vastly reduce the skill level required and increase availability of such services. Regards Greg Dr Greg Low SQL Down Under Pty Ltd +61 419201410 ________________________________ From: Scott <scott.barnes@gmail.com<mailto:scott.barnes@gmail.com>> Sent: Tuesday, July 30, 2024 8:37:40 PM To: ozDotNet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> Cc: Tom Gao <tom@tomgao.com<mailto:tom@tomgao.com>>; David Connors <david@connors.com<mailto:david@connors.com>>; Dr Greg Low <greg@sqldownunder.com<mailto:greg@sqldownunder.com>> Subject: Re: AI omfg. i forgot about this list :D It may replace Business Analyst and partially Architects one day. I'm currently dealing with a constant battle between Developers and Business Analysts over daily ownership of requirements that address solutioning issues. The circuit breaker has been using the entire Jira tickets for a project, feeding them to AI Crew Agents (https://github.com/crewAIInc/crewAI), and having them work overnight on creating an end-to-end technical specifications document based on the existing code base (legacy) combined with the proposed user stories. I also use some TOGAF templates to test a concise formatting option. The results are remarkably professional and accurate compared to the human-centric approach used as a baseline comparison. When multiple AI agents share a context and each has a unique role, it opens up different approaches to communication issues or problem-solving in terms of legacy/reverse engineering mixed with future technical solutioning. It still often hallucinates in critical areas of thought, but i'd say it accelerates the work required here by about 60-70% at the very least (much like it does with C++ code in Unreal Engine, it gives you the breadcrumb trail at the very least to go shape towards a direction you need to head but won't make the game for you sort of thing). Another practical use case at the moment is in the travel industry. Often, the GDS systems have an open text field for additional but pertinent information for accommodations (e.g., key is under a plant, towels need to be washed, etc.), and there is no uniform standard for storing that information (because the travel industry is rarely ever consistent). Using AI, we can feed it a series of specific templates and ask it to match the criteria based on the types of remarks (think of it as a cascade of chaining different types). I don't think it's currently anywhere near a stage where it's a job replacement thing, but it's definitely getting there and shaping data towards "source of truth" practical usage. On Tue, Jul 30, 2024 at 7:26 PM Dr Greg Low via ozdotnet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> wrote: No old friend at Blackburn Dr Greg Low SQL Down Under Pty Ltd +61 419201410 ________________________________ From: Tom Gao <tom@tomgao.com<mailto:tom@tomgao.com>> Sent: Tuesday, July 30, 2024 6:54:08 PM To: ozDotNet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> Cc: David Connors <david@connors.com<mailto:david@connors.com>>; Dr Greg Low <greg@sqldownunder.com<mailto:greg@sqldownunder.com>> Subject: Re: AI Hi Greg, funny that I was also at my ophthalmologist this morning there's not many around were you around randwick by any chance ? lol On Tue, Jul 30, 2024 at 6:40 PM Dr Greg Low via ozdotnet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> wrote: I should add that the reason he raised it was being literally scared about the future. And not just for himself, the future of work is suddenly becoming apparent to him. Regards Greg Dr Greg Low SQL Down Under Pty Ltd +61 419201410 ________________________________ From: David Connors via ozdotnet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> Sent: Tuesday, July 30, 2024 5:59:37 PM To: ozDotNet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> Cc: David Connors <david@connors.com<mailto:david@connors.com>> Subject: Re: AI Doesn't surprise me. We recently bought a new house and I was trying to make sense of all of the town plan documents during the settlement period. I just uploaded it all to ChatGPT and asked for an explanation of everything including impacts of easements etc. I sent the chat to my town planner and he was pretty shocked at how good it was. It is not going to delete professions but unless you're onboard with it you won't be able to meet the performance expectations of your customers / employers. On Tue, 30 Jul 2024 at 16:55, Dr Greg Low <greg@sqldownunder.com<mailto:greg@sqldownunder.com>> wrote: Was at an eye specialist this morning for a check-up. When we were finished, he wanted to talk about AI. He showed me a system they've started using. It combines input from around six separate technical systems that they use, reads all of the patient history, clinical notes, and compares the captured images across all the years I've been to see him. As he writes his clinical notes in for the day, it wrote a summary of unbelievably high quality. He told me how it was way better than anything he could ever do, particularly the way it reviewed everything from the past and compared it (in detail) to the current. He was genuinely concerned about what sort of job he's going to have in future. It's not replacing him yet, but it's completely changed his ability to work at a much higher level. Regards, Greg Dr Greg Low 1300SQLSQL (1300 775 775) office | +61 419201410 mobile SQL Down Under | Web: https://sqldownunder.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__sqldownunder.com_&d=DwMFAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=2rgtwrXggQFZiZbisdwDooYFalucb-vLhjG0McaanBZKn0UVuognuHqfHnjp2AVc&m=I23jyX4AKIv9q2x7A3CQAer9PGCjq8R6DwW7BE1IAhZ1JbigKMrMPRCjs6AqW7h3&s=o3oFliHztOF8D9Nbqaa7KQdqC-zkQNXWl4IqnEG58Wc&e=> | About Greg: https://about.me/greg.low<https://urldefense.proofpoint.com/v2/url?u=https-3A__about.me_greg.low&d=DwMFAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=2rgtwrXggQFZiZbisdwDooYFalucb-vLhjG0McaanBZKn0UVuognuHqfHnjp2AVc&m=I23jyX4AKIv9q2x7A3CQAer9PGCjq8R6DwW7BE1IAhZ1JbigKMrMPRCjs6AqW7h3&s=NsAibgiqfCxsyc8m2DBKogKQcs3OqE3mkyCjmpoYxTk&e=> From: David Connors via ozdotnet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> Sent: Tuesday, July 30, 2024 2:17 PM To: ozDotNet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> Cc: David Connors <david@connors.com<mailto:david@connors.com>> Subject: Re: AI On Sun, 28 Jul 2024 at 23:16, Tom Gao via ozdotnet <ozdotnet@ozdotnet.com<mailto:ozdotnet@ozdotnet.com>> wrote: I wanted to get some feedback from you guys last time michael said maybe about 1-2% in improvement in usefulness to a seasoned dev. I've had ongoing robust conversations. Because I feel the benefits are completely overstated in general (open to being corrected). I don't think the benefit is overrated and we see huge value in the use of AI across all of the dev work we do. I think you're possibly looking at it glass half empty instead of glass half full. The question is not whether the AI is smarter than you, can write better code than you - the question is whether you get a great productivity boost in day to day activities. Co-pilot smashes boring drudge work like making constructors out of the park. You can use ChatGPT to break the back of time consuming but easy work - for example, upload a CSV and then ask it to write you the DDL for a table to store that and a stored procedure to do CRUD operations etc. It churns it out instantly. If you asked me to write an UPSERT statement in T-SQL or create a user in MySQL I would have to Google it. They're tasks that are easy, but also tasks that I seldom do so don't have the syntax front of mind. I can just leave out the middle-man and get the AI to do it for me. The most senior tech lead in my team believe it's a negative benefit. As in it can't understand complex business analyst requirements, nor debug codes, nor find / correct defects. All of our PRs are reviewed by AI before humans touch them (via Code Rabbit). People reviewing PRs are generally more senior, more expensive and time poor. Code Rabbit produces *really* good summaries of what is in the PR and the implications. It provides a lot of low level commentary that is useful to the dev's themselves before asking for a review. You can talk back and forth with Code Rabbit in the comments under the PR in GitHub and ask for suggestions etc. The effort to prompt engineer and also to correct what's generated is also understated. What sort of things are you asking it to do? If you're trying to get an LLM to write conceptually complex things then that might be where you have friction. Here's an example of a CodeRabbit review for a simple change done today in one of our systems together with example of dev conversation. The reviews of complex changes (I could not find a good example without internal information) are astonishingly good and the back and forth with the bot can be very nuanced (arguing the point on why there isn't actually a race condition in some code because it isn't considering this or that). This level of review / suggestion / fix cycle happens before a senior considers the PR. The poems are a nice touch :) [cid:ii_19103a3fed84cff312] Conversation on that change with CodeRabbit: [cid:ii_19103a3fed85b16b21] -- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/ -- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/

On Tue, 30 Jul 2024 at 19:02, Tom Gao via ozdotnet <ozdotnet@ozdotnet.com> wrote:

Don't know David, back in my days of coding if I couldn't write T-SQL and I had to google, I wouldn't get a job... maybe devs are different these days?

Times have changed. If you ask me for most common / old school DDL or DML I can rattle it off pretty easily as it is in my muscle memory from uni days... don't ask for an upsert or pivot statement though - or any of the new BI stuff. Also, new DBMS are coming up all the time and everything is the same and different - AI really helps there.

I call it my "low IQ assistant". If I have some menial task that I don't feel like coding, I ask the AI to do it for me and it's usually pretty good - except that you do need to check everything! You could ask a junior to do the same thing. I have tested a number of AIs with a bunch of tech questions and the one that got by far the highest score was Bing Chat - that said I didn't have access to GPT-4 and Bing Chat is essentially GPT-4 but for free. It's now called Copilot. I also use it for assisting in comprehending business requirements. I take a doco and ask it to group common themes and produce summaries of the requirements. Like most developers, my attention would be struggling otherwise. I can also ask it to act like a BDD expert and produce Gherkin statements for the testers. It helps a lot when I get stuck on a problem. I often get a much better answer if I ask Bing Chat than if I try to Google it. That could, of course, be a consequence of how bad Google has become at providing an answer. Sometimes other people in the team have issues and they come to me with a much more sophisticated problem. This might mean that I don't even know the context of the issue they are having, but I can interrogate Bing Chat and it will give me a response that is quite helpful in many cases. It helps when I want to get started on a new problem. I can ask it what I think I need to know and it often returns answers that help me fine tune what I am trying to do. Sometimes that requires multiple interactions. If I'm looking for a new library to solve a problem, I can ask it for advice on what to look at. It is often helpful, but sometimes makes wrong assumptions about what I am trying to achieve. The important thing is it gives me an idea of some libraries to consider. Finally, there comes a point at which its value drops and I am getting better at detecting when that happens and...head to stack overflow.

On Fri, Feb 23, 2024 at 12:25 PM Dr Greg Low via ozdotnet < ozdotnet@ozdotnet.com> wrote:

There were other things I should have mentioned.

The new PowerPoint co-pilot where you just say “Prepare me a presentation about what’s in xyz.docx” is pretty amazing.

I’ve used ChatGPT to rewrite marketing blurb for various things. It does that very well. However, I’ve asked it to improve a paragraph of writing, and find that something like the Hemmingway editor does a far superior job.

In Teams, having the AI tool write a summary of what just happened in a meeting is pretty stunning.

We are going to just be using these tools all day long.

Regards,

Greg

Dr Greg Low

1300SQLSQL (1300 775 775) office | +61 419201410 mobile

SQL Down Under | Web: https://sqldownunder.com <https://urldefense.proofpoint.com/v2/url?u=https-3A__sqldownunder.com_&d=DwM...> | About Greg: https://about.me/greg.low <https://urldefense.proofpoint.com/v2/url?u=https-3A__about.me_greg.low&d=DwM...>

*From:* Dr Greg Low *Sent:* Friday, February 23, 2024 12:11 PM *To:* ozDotNet <ozdotnet@ozdotnet.com> *Cc:* Tom Gao <tom@tomgao.com> *Subject:* RE: AI

Hi Tom,

For me, it depends what you want it to do. It certainly can appear to help someone who’s new to an area.

For most code writing, I’ve been pretty underwhelmed. As an example, if I ask it to write SQL, I get a very poor outcome. It will use old deprecated views instead of the current system views (that have been around for a decade), and often does things in a convoluted way.

What I have been impressed with, is how it can help you understand acronyms, etc. Quite amazing. I’ve also been pretty impressed with using it go generate some test data, including in multiple languages. And the test data is fairly believable. If I ask it for family names, and I also ask for Chinese, it does pick common Chinese family names in the test output. That’s pretty impressive.

It can do a reasonable job of things like “here’s some DAX code, can you simplify it?” It often can. Or “here’s a regular expression, can you explain what it does?” and it does that just fine. I’ve seen people happily using it to explain code that they don’t understand, or to (sort of) document some code.

But it also is so confident on things, yet so wrong. I gave it a 25 question baseball umpire test the other day. It was 100% confident sounding, but 40% correct. The weird thing is that some of the questions that it got right, are things that new human umpires often get wrong. Yet for simpler questions, it would say that something legal is illegal.

It’s certainly interesting, but it’s very much a work in progress. It will be part of our futures.

Regards,

Greg

Dr Greg Low

1300SQLSQL (1300 775 775) office | +61 419201410 mobile

SQL Down Under | Web: https://sqldownunder.com <https://urldefense.proofpoint.com/v2/url?u=https-3A__sqldownunder.com_&d=DwM...> | About Greg: https://about.me/greg.low <https://urldefense.proofpoint.com/v2/url?u=https-3A__about.me_greg.low&d=DwM...>

*From:* Tom Gao via ozdotnet <ozdotnet@ozdotnet.com> *Sent:* Friday, February 23, 2024 11:58 AM *To:* ozDotNet <ozdotnet@ozdotnet.com> *Cc:* Tom Gao <tom@tomgao.com> *Subject:* AI

Hi guys, I haven't posted in a few years and haven't been on the tools for a long time now as well. I'm on a panel on a digital conference coming up in march. We had a pre meeting today and the topic of AI came up. Two of the panelist said cited CBA and Westpac using AI and were able to save 30% on development effort.

Personally I just finished an AI course my view is quite the opposite. My personal opinion of the generative AI space and AI in general having spent time with the academics is that the benefits are significantly over inflated.

I want to get some other opinions if you are seeing any significant benefit and that I may be just out of touch or not aware.

Thanks,

Tom -- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/

It has helped me ramp up on a new job far faster than I've ever ramped up before, and without having to ask questions of key busy staff. I've been able to ask it to explain terminology as well in the sector I'm working in. By not having to bother key people with what must otherwise appear to them as trivial questions, it increases my credibility. I only have to ask them the more complicated domain related questions. I'm also mainly a SQL Server person. My current job uses Oracle and MySQL. I can ask it to show me examples of how to perform tasks that I do in SQL Server but in Oracle or MySQL. I can ask it to explain some Oracle procs/functions. I can also provide it with code in one language and ask it to change it to another language. We use BuildKite at work. I have no interest in learning BuildKite. I have given it a BuildKite script and asked it to translate it to a GitHub Action script. It gets pretty close. Yes, I still need to debug the script, but it's far easier if the general jist of the job is already there. I have found that people most turned off AI are the ones that tried it in the early days and haven't tried GPT-4, or are stuck on the free version. I can understand people walking away after using GPT3.5. It simply gave garbage answers at least half of the time, and so became a time waster. Some people are stuck on the first AI they find most effective. I will most likely suffer from this problem. Until someone proves that another AI is better than the one I am using, I will find it difficult to move on. People at work that tried GPT3.5 have pretty much refused to use Copilot. I think that's a mistake, as it's only going to get better as time goes on and the people willing to keep moving forward are the ones that will get the most benefit from productivity improvements, potentially making the others look bad or slow. Hope that helps, enough from me for now! On Fri, Feb 23, 2024 at 12:40 PM Tony Wright <tonywr71@gmail.com> wrote:

I have also asked it to perform code reviews on code. It's pretty good at refactoring code, and giving you pointers on improvements. When staff raise pull requests, I have my own ideas, but I can also get Copilot to provide suggestions. (I also have Github Copilot, which work provides) It's very good for help in understanding what historic code does too.

In general I find it an invaluable tool. It won't be replacing us any time soon, but people that aren't using it will be less productive. Does it make me 30% more efficient? I don't "feel" it, but there's probably a lot of research time I am saving, and perhaps I don't even realise all the benefits it's giving me.

On Fri, Feb 23, 2024 at 12:33 PM Tony Wright <tonywr71@gmail.com> wrote:

...

I call it my "low IQ assistant". If I have some menial task that I don't feel like coding, I ask the AI to do it for me and it's usually pretty good - except that you do need to check everything! You could ask a junior to do the same thing.

I have tested a number of AIs with a bunch of tech questions and the one that got by far the highest score was Bing Chat - that said I didn't have access to GPT-4 and Bing Chat is essentially GPT-4 but for free. It's now called Copilot.

I also use it for assisting in comprehending business requirements. I take a doco and ask it to group common themes and produce summaries of the requirements. Like most developers, my attention would be struggling otherwise. I can also ask it to act like a BDD expert and produce Gherkin statements for the testers.

It helps a lot when I get stuck on a problem. I often get a much better answer if I ask Bing Chat than if I try to Google it. That could, of course, be a consequence of how bad Google has become at providing an answer. Sometimes other people in the team have issues and they come to me with a much more sophisticated problem. This might mean that I don't even know the context of the issue they are having, but I can interrogate Bing Chat and it will give me a response that is quite helpful in many cases.

It helps when I want to get started on a new problem. I can ask it what I think I need to know and it often returns answers that help me fine tune what I am trying to do. Sometimes that requires multiple interactions. If I'm looking for a new library to solve a problem, I can ask it for advice on what to look at. It is often helpful, but sometimes makes wrong assumptions about what I am trying to achieve. The important thing is it gives me an idea of some libraries to consider.

Finally, there comes a point at which its value drops and I am getting better at detecting when that happens and...head to stack overflow.

...

On Fri, Feb 23, 2024 at 12:25 PM Dr Greg Low via ozdotnet < ozdotnet@ozdotnet.com> wrote:

...

There were other things I should have mentioned.

The new PowerPoint co-pilot where you just say “Prepare me a presentation about what’s in xyz.docx” is pretty amazing.

I’ve used ChatGPT to rewrite marketing blurb for various things. It does that very well. However, I’ve asked it to improve a paragraph of writing, and find that something like the Hemmingway editor does a far superior job.

In Teams, having the AI tool write a summary of what just happened in a meeting is pretty stunning.

We are going to just be using these tools all day long.

Regards,

Greg

Dr Greg Low

1300SQLSQL (1300 775 775) office | +61 419201410 mobile

SQL Down Under | Web: https://sqldownunder.com <https://urldefense.proofpoint.com/v2/url?u=https-3A__sqldownunder.com_&d=DwM...> | About Greg: https://about.me/greg.low <https://urldefense.proofpoint.com/v2/url?u=https-3A__about.me_greg.low&d=DwM...>

*From:* Dr Greg Low *Sent:* Friday, February 23, 2024 12:11 PM *To:* ozDotNet <ozdotnet@ozdotnet.com> *Cc:* Tom Gao <tom@tomgao.com> *Subject:* RE: AI

Hi Tom,

For me, it depends what you want it to do. It certainly can appear to help someone who’s new to an area.

For most code writing, I’ve been pretty underwhelmed. As an example, if I ask it to write SQL, I get a very poor outcome. It will use old deprecated views instead of the current system views (that have been around for a decade), and often does things in a convoluted way.

What I have been impressed with, is how it can help you understand acronyms, etc. Quite amazing. I’ve also been pretty impressed with using it go generate some test data, including in multiple languages. And the test data is fairly believable. If I ask it for family names, and I also ask for Chinese, it does pick common Chinese family names in the test output. That’s pretty impressive.

It can do a reasonable job of things like “here’s some DAX code, can you simplify it?” It often can. Or “here’s a regular expression, can you explain what it does?” and it does that just fine. I’ve seen people happily using it to explain code that they don’t understand, or to (sort of) document some code.

But it also is so confident on things, yet so wrong. I gave it a 25 question baseball umpire test the other day. It was 100% confident sounding, but 40% correct. The weird thing is that some of the questions that it got right, are things that new human umpires often get wrong. Yet for simpler questions, it would say that something legal is illegal.

It’s certainly interesting, but it’s very much a work in progress. It will be part of our futures.

Regards,

Greg

Dr Greg Low

1300SQLSQL (1300 775 775) office | +61 419201410 mobile

SQL Down Under | Web: https://sqldownunder.com <https://urldefense.proofpoint.com/v2/url?u=https-3A__sqldownunder.com_&d=DwM...> | About Greg: https://about.me/greg.low <https://urldefense.proofpoint.com/v2/url?u=https-3A__about.me_greg.low&d=DwM...>

*From:* Tom Gao via ozdotnet <ozdotnet@ozdotnet.com> *Sent:* Friday, February 23, 2024 11:58 AM *To:* ozDotNet <ozdotnet@ozdotnet.com> *Cc:* Tom Gao <tom@tomgao.com> *Subject:* AI

Hi guys, I haven't posted in a few years and haven't been on the tools for a long time now as well. I'm on a panel on a digital conference coming up in march. We had a pre meeting today and the topic of AI came up. Two of the panelist said cited CBA and Westpac using AI and were able to save 30% on development effort.

Personally I just finished an AI course my view is quite the opposite. My personal opinion of the generative AI space and AI in general having spent time with the academics is that the benefits are significantly over inflated.

I want to get some other opinions if you are seeing any significant benefit and that I may be just out of touch or not aware.

Thanks,

Tom -- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/