Hi Tom et al, Coincidentally, my CoPilot trial ends today and I've decided to let it continue at $US10/month. So my AI qualifications now begin with one month of GitHub Copilot. Is this particular AI useful for developers? Urrrr...sort of...I've got mixed feelings, but I'm going to let it run for some more months and see how it goes. People in demos everywhere, even many of the top boffins have it running in their videos, so if it's good enough for Mads and Anders then I like what they like. Sometimes Copilot throws out suggestions that are jaw-droppingly psychically predictive and I stare in disbelief before hitting tab. Other times it produces something completely irrelevant that was snipped from a different project I was previously working on. The former seems to outnumber the latter, so that's why I'm continuing the subscription. As Tom hints, I also probably get a 1-2% boost in productivity thanks to Copilot (well, I will when my reflexes adapt to stop having keyboard races with it). I feel the name "AI" is a bit generous, as it's clearly just trawling through vast amounts of code and samples, analysing it and spitting fragments back out at you based on patterns and context. There are fabulous algorithms at work finding patterns, but is this "AI"? When I first saw the movie *2001: A Space Odyssey* in 1969 I was fascinated by HAL (I was clearly destined to work in IT!), and now 55 years later I'd like to be able to sit down with HAL and explain some complex business requirement to him, converse like professionals and weigh-up the pros and cons of different platforms and languages based upon his world of experience and get him to code like a 1000x developer and generate a complete working skeleton of the required product. When will I see that? I'm a bit disappointed so far. Oh well, back to coding with my Copilot buddy. *Greg K* On Sun, 28 Jul 2024 at 23:19, Tom Gao via ozdotnet <ozdotnet@ozdotnet.com> wrote:
Hey Guys, it's been awhile since I've asked the question about the usefulness of AI in our development world - for a seasoned senior developer for a complex digital transformation project. Since then I've met the microsoft board member on OpenAI and have completed an AI course at Stanford and seen the latest and greatest at the most advanced world leading AI lab. To be honest I think this AI thing is going to be 10 years away to be truly useful as it is hyped to be. But this is just a personal opinion.
I wanted to get some feedback from you guys last time michael said maybe about 1-2% in improvement in usefulness to a seasoned dev. I've had ongoing robust conversations. Because I feel the benefits are completely overstated in general (open to being corrected). The most senior tech lead in my team believe it's a negative benefit. As in it can't understand complex business analyst requirements, nor debug codes, nor find / correct defects. The effort to prompt engineer and also to correct what's generated is also understated. Plenty of tools already do what chatgpt/co-pilot can do. In this case we've experimented with github copilot which claims to provide 55% efficiency in coding.
What's everyone's thoughts since Feb this year? - how useful is it in real world dev tasks, building complex transformation projects from BA requirements?
Thanks in advance, Tom
On Sat, Feb 24, 2024 at 9:21 PM Michael Ridland via ozdotnet < ozdotnet@ozdotnet.com> wrote:
If you’re a seasoned practitioner in the language it’s probably sitting around 1-2%, because it can help with some code conversions. Copilot code completions are amateur. For .net resharper/rider etc is actually a much better option than copilot, practical use of that is probably around 15% productivity increase.
Copilot does help if you don’t know the language or technology, which maybe commbank staff are not skilled up enough? ;)
On Sat, 24 Feb 2024 at 6:27 pm, DotNet Dude via ozdotnet < ozdotnet@ozdotnet.com> wrote:
I’m not a subscriber to read the entire article but 30% is a load of BS at this stage
On Sat, 24 Feb 2024 at 15:18, Tom Gao via ozdotnet < ozdotnet@ozdotnet.com> wrote:
Here's the article.
https://www.afr.com/technology/commbank-claims-ai-is-already-making-it-work-...
So what does everyone think about the claim that 30% efficiency? ie to me as head of IT instead of hiring 10 developers to build applications now only 7 is required. To me that just doesn't seem realistic.
On Fri, Feb 23, 2024 at 3:04 PM Tom Rutter via ozdotnet < ozdotnet@ozdotnet.com> wrote:
On Fri, 23 Feb 2024 at 12:32, Dr Greg Low via ozdotnet < ozdotnet@ozdotnet.com> wrote:
Worse, in the baseball umpiring exam I mentioned, someone said to me “isn’t that what you’d get if you just asked a whole lot of fans about the rules rather than asking umpires?”
There’s probably something important about that. How does it know which of the material it was trained on is valid?
Yep that’s the thing, it doesn’t actually know what is “valid”. From what I understand it just gets “pushed” in certain directions during training and not in all cases obviously.
My experience with things like ChatGPT so far basically brings it down to a better Google really. Early days
Regards,
Greg
Dr Greg Low
1300SQLSQL (1300 775 775) office | +61 419201410 mobile
SQL Down Under | Web: https://sqldownunder.com <https://urldefense.proofpoint.com/v2/url?u=https-3A__sqldownunder.com_&d=DwM...> | About Greg: https://about.me/greg.low <https://urldefense.proofpoint.com/v2/url?u=https-3A__about.me_greg.low&d=DwM...>
*From:* mike smith via ozdotnet <ozdotnet@ozdotnet.com>
*Sent:* Friday, February 23, 2024 12:19 PM *To:* ozDotNet <ozdotnet@ozdotnet.com> *Cc:* mike smith <meski.oz@gmail.com> *Subject:* Re: AI
"old system views"
That makes me wonder if it has any way of differentiating between something it found from a decade ago to more recent data.
Mike
On Fri, 23 Feb 2024, 11:43 Dr Greg Low via ozdotnet, < ozdotnet@ozdotnet.com> wrote:
Hi Tom,
For me, it depends what you want it to do. It certainly can appear to help someone who’s new to an area.
For most code writing, I’ve been pretty underwhelmed. As an example, if I ask it to write SQL, I get a very poor outcome. It will use old deprecated views instead of the current system views (that have been around for a decade), and often does things in a convoluted way.
What I have been impressed with, is how it can help you understand acronyms, etc. Quite amazing. I’ve also been pretty impressed with using it go generate some test data, including in multiple languages. And the test data is fairly believable. If I ask it for family names, and I also ask for Chinese, it does pick common Chinese family names in the test output. That’s pretty impressive.
It can do a reasonable job of things like “here’s some DAX code, can you simplify it?” It often can. Or “here’s a regular expression, can you explain what it does?” and it does that just fine. I’ve seen people happily using it to explain code that they don’t understand, or to (sort of) document some code.
But it also is so confident on things, yet so wrong. I gave it a 25 question baseball umpire test the other day. It was 100% confident sounding, but 40% correct. The weird thing is that some of the questions that it got right, are things that new human umpires often get wrong. Yet for simpler questions, it would say that something legal is illegal.
It’s certainly interesting, but it’s very much a work in progress. It will be part of our futures.
Regards,
Greg
Dr Greg Low
1300SQLSQL (1300 775 775) office | +61 419201410 mobile
SQL Down Under | Web: https://sqldownunder.com <https://urldefense.proofpoint.com/v2/url?u=https-3A__sqldownunder.com_&d=DwM...> | About Greg: https://about.me/greg.low <https://urldefense.proofpoint.com/v2/url?u=https-3A__about.me_greg.low&d=DwM...>
*From:* Tom Gao via ozdotnet <ozdotnet@ozdotnet.com> *Sent:* Friday, February 23, 2024 11:58 AM *To:* ozDotNet <ozdotnet@ozdotnet.com> *Cc:* Tom Gao <tom@tomgao.com> *Subject:* AI
Hi guys, I haven't posted in a few years and haven't been on the tools for a long time now as well. I'm on a panel on a digital conference coming up in march. We had a pre meeting today and the topic of AI came up. Two of the panelist said cited CBA and Westpac using AI and were able to save 30% on development effort.
Personally I just finished an AI course my view is quite the opposite. My personal opinion of the generative AI space and AI in general having spent time with the academics is that the benefits are significantly over inflated.
I want to get some other opinions if you are seeing any significant benefit and that I may be just out of touch or not aware.
Thanks,
Tom
-- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
-- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
-- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
-- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
-- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
-- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/
-- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/