Showing posts with label Code Freeze. Show all posts
Showing posts with label Code Freeze. Show all posts

Tuesday, February 13, 2024

Code Freeze 2024 - AI for SE for AI

Code Freeze 2024 - AI for SE for AI

I took notes at the U of Minnesota Code Freeze event put on yearly by the U of MN Comp Sci department.  It's a one-day event and the topics have changed every year.  I honestly don't know how long I've been going to the event.  Based on their list, a long time, because I know I was Redesigning Agility, and I don't see the one related to offshoring in there, so that must have been during 1-4.  A quick scan of the blog shows a post for 2008, so at least that far back.  Quality, or at least applicability to me, varies and the topic isn't always aligned with how interesting the event is.  For instance, the 2015 event on Security was one of my favorites, although day to day I don't worry about security too much anymore outside of Mend scans [previously Blackduck scans], despite once having done enough FEDRAMP and SOC 2 TYPE 2 work to be able to present to a class at the U of MN Duluth on the topic.  But the Code Freeze was more about hacking cars and airport scanners.

Some of these notes will be pretty loose without explanation, but per the intro, it stayed fairly high level in my opinion.

  • 2006: Inaugural Edition
  • 2007: Second Edition
  • 2008: Third Edition
  • 2009: Maximizing Developer Value 
  • 2010: Redesigning Agility 
  • 2011: Software Testing for the 21st Century 
  • 2012: Continuous Delivery 
  • 2013:​​​​​ Software Anthropology 
  • 2014: Big Data 
  • 2015: Security 
  • 2016: Disruptive Innovation 
  • 2017: Software Visualization 
  • 2018: Microservice Architectures 
  • 2019: Machine Learning 
  • 2020: Observability 
  • 2021: Humane Engineering
  • 2022: Developer Experience
  • 2023: Tech Resilience

Because it's applicable, here's an article from later: https://medium.com/artificial-corner/i-tried-multiple-ai-coding-assistants-these-are-the-best-bb4d6a735fc1 


It was an exciting day.  Had breakfast at Al’s Breakfast at 6 a.m. before the conference. Well worth getting up that early. As a bonus, parking over there, when it's not below 0F, means I can park for free and get in a good walk before the conference starts.



As I was walking to the conference building, there were an inordinate number of police SUVs parked across from Moos tower and driving along the central road.  Turns out there was a shooting threat resulting in staff being told to stay home and all doors locked/monitored.  The conference went on, but folks were definitely a little nervous.



The one thought that binds them all - try all the new tools.  Just start using as many as you can and understand what they do and how they’re changing.  There is a non-stop flow entering the marketplace.


  • unchecked

    Do a copilot compare on the team?  Have everyone pick one [a different one] to try out and do a five-ten minute presentation on how it went [with some sort of standardized points].  Ask ChatGPT to give me a summarized list?


Chat GPT only cares about January 2022 and earlier…

  • TabNine:

    • Strengths:

      • Uses machine learning to predict code completions efficiently.

      • Supports multiple programming languages.

    • Weaknesses:

      • Requires a subscription for advanced features.

  • Kite:

    • Strengths:

      • AI-powered code completions.

      • Provides relevant documentation and examples.

    • Weaknesses:

      • Some users may find the AI suggestions intrusive.

  • Visual Studio IntelliCode:

    • Strengths:

      • Uses machine learning for personalized code suggestions.

      • Integrates well with Visual Studio.

    • Weaknesses:

      • Limited language support compared to some other tools.

  • Codota:

    • Strengths:

      • AI-driven code completions.

      • Supports various IDEs and editors.

    • Weaknesses:

      • Free version has limitations, and full features may require a subscription.

  • DeepCode:

    • Strengths:

      • Analyzes code patterns to find bugs and suggest improvements.

      • Integrates with various IDEs.

    • Weaknesses:

      • May not cover as many languages as some competitors.

  • CodeIntel:

    • Strengths:

      • Provides code intelligence for multiple languages.

      • Open-source and customizable.

    • Weaknesses:

      • Requires some configuration for optimal use.

  • Sourcegraph:

    • Strengths:

      • Offers code intelligence across repositories.

      • Integrates with popular code editors.

    • Weaknesses:

      • Can be resource-intensive for larger codebases.

  • GitHub Copilot:

    • Strengths:

      • Generates whole lines or blocks of code based on comments.

      • Built-in collaboration with VS Code.

    • Weaknesses:

      • May produce incorrect or insecure suggestions.

  • CodeRush:

    • Strengths:

      • Includes code generation and refactoring tools.

      • Visual Studio integration.

    • Weaknesses:

      • Learning curve for mastering all features.

  • Eclipse Code Recommenders:

    • Strengths:

      • Context-aware code recommendations.

      • Supports Java development.

    • Weaknesses:

      • Limited language support compared to some other tools.

Unrelated things Scott was thinking while at the conference:

  • unchecked

    Can I AI generate the Tinkerpop in the D and D Node Database demo that’s out there? Can you use it to to generate the synthetic data and the queries?

  • unchecked

    Is there a new Tech Radar out yet? Thoughtworks’ last was September 2023

  • unchecked

    Can i generate an interview outline based on my write up of skills interns should learn and using job reqs for the company or our everything to know about the team writeup?  Can we generate how to learn our team-specific tech using AI?

Keynote Presentation - Andreas Sjostrom [Capgemini] - Igniting the Future: Enterprise AI’s Quantum Leap and the Dawn of Artificial Minds

  • Approached from a C-suite landscape, and then from other facets/perspectives.

  • “If you have family in France, warn them that the next call from their son or daughter will be a fake.” [funny quote for Clement]

  • There are tools in all the spaces - if you like one in your core area, look for agents in other areas as well:

    • Data Herald - natural language to SQL

    • ML Twist - data labeling

    • He likes Kodezi better than MS Copilot

    • Language chain

  • Agentive AI - a general term for all the helpers.

  • Sent myself the images he used.



Didn’t really address cost.

Raj Subrameyer - ChaiLatte Consulting - Building Better Software Faster: The Generative AI Advantage


Markov [70s] > n-grams [90s] > neural networks [2000s] > transformers [2010s] > now


Chat GPT, Embedded models, code models, DALLE.


  1. Code Generation [Github Copilot, Amazon Codewhisperer, VS Code integration for most]

  2. Documentation [Komment, MintLify for Typescript, Sourcegraph, Tabnine]

  3. Testing: single location strategy evolved to multi location strategy.  Autonomous testing > test in production and use production to generate the tests via AI.  What are the customers actually doing?  Appli Tools, Blazemeter.

  4. Ethics/Challenges





Matt Schillerstrom - Harness - Where should I infuse AI in the SDLC?

  • From Monticello.  Went to St. John’s University.  Worked at the Monti nuclear plant.  We shared teachers and friends [or at least siblings of friends].

  • CI/CD shift left security.  Lot of waste in this space.

  • Artificial versus “Augmented”.

  • Talked IDPs - Internal Dev Portals.  More and more common.  Alteryx probably needs one [I don’t think anything we have officially counts.  I think one - the Spotify version - was demoed at a learning event]

  • Tools for the CI/CD space: Starroper, Mistral AI, Sophia.  There are K8s helpers.

  • Is AI for these tools centralized at the org level or distributed to the teams? [think how cost has been centralized, security has been centralized, etc]

  • AI is creating an explosion of code that CI/CD has to deal with.


Arun Batchu - Gartner - Survive and Thrive in the Age of Generative AI: How Software Engineers Can Ride the Wave of Foundation Models


  • Don’t forget, AI can always analyze for you.  It doesn’t have to do the thing, it can help you with the steps to understand how to get to the thing [treat it like a conversational friend]

  • Generate synthetic test data

  • “Coding assistants are force multipliers”

  • Use AI to teach you - I am a Python dev, tell me how to learn typescript…

  • Embrace coding assistants

  • There’s a Google Notebook LLM that people should try

  • “Multimodal prompt”

  • Amazon Bedrock

  • Google Extensions

  • ChatGPT app store [like 800000 entries already]

  • Chat GPT will now tap into code writing if you ask it a question that necessitates that - an example, I want the 100th fibonacci number.  Behind the scenes, it’s coding.

  • There are writing tools.  Use them.

  • Adapt foundation models and use foundation model agents.

  • Don’t forget, YOU are a manual job and AI is accelerating automation to eliminate manual jobs [so how do you fit into the ecosystem and how it’s changing].


David Espindola - Brainyus - You, Software Warrior, In the Future of Artificial Intelligence

Maybe my least favorite presentation.  He wrote two books to talk about exponential technology curve and inflection points..  He highlighted like nine trends, but did not highlight the interaction of those trends as an overall trend [I’m a little biased because my master’s thesis on dystopias covered much of what he’s covering here]


Disruptors.


Jeffrey Fougere and Andrew Nieuwsma - Hewlett Packard - Using ChatGPT for Software Engineering



  • Practical examples - they had a ethics of AI app they’d written using AI that prompted with questions [natural language] to ensure someone was taking a use case to all logical [and unanticipated] scenarios.

  • Talked through the ethics: other companies may have a problem with your AI [I have an example, as the last company I worked for had to code around AI exclusions including Bank of America.  We had full opt out and individual recommendation AI/ML opt out - at every level].

  • Lots of walking through prompts.

  • Ethics, bias, hallucinating, legal risks.

  • “Explain to me how this works…” prompt.

Sunday, January 19, 2020

Code Freeze 2020 - Observabillity

Sorry for the loose nature of the notes rather than a good writeup, but I wanted to get things collated so I can work with them.  Good conference.  I ate at Al's for breakfast and Hong Kong Noodles for lunch.  And I realized I take the green line over to the U of MN for lunch anytime I'm downtown at work, which didn't occur to me these last six months.  If I had my bicycle, it's even a short ride that way across the campus bridge.  I need to get my urban on.  Not as much practical knowledge at this one (for me) as some of the past events, but that means I can focus on the few things I think have practical value rather than being all over the place.

Observability and the Glorious Future - Charity Majors (Honeycomb.io)

  • O'Reilly Database Reliability Engineering (November 2017: http://shop.oreilly.com/product/0636920039761.do)
  • How often do you deploy.  How long, how often do you fail, recovery time - the basics.
  • Hires for communication skills (initial tech interview is to get them talking at the in person).  "Empowered to do their jobs". 
  •  "How do I know if it breaks?" - all changes, all features
  • "Serverless was a harbinger.  Deployless is coming."
  • Developers (senior+) should amplify the hidden costs.
  • Team happiness = customer happiness (Steve says this too)

Observability in Big Analytics - Bonnie Holub, Teradata

50 Years of Observability - Mary Poppendieck

  • What is the equivalent of metal fatigue in software?  Operator fatigue. >> e.g. what Steve pushes that a focus on PIs is important.
  • Talked planes, bridges, three mile island
  • She likes the Control series by Brian out on Youtube....they're deep: https://www.youtube.com/channel/UCq0imsn84ShAe9PBOFnoIrg
  • Observable - all critical states known from system outputs
  • Observable is at war with complexity.
  • Controllable activator - sensor can get back to a set state in a set time.
  • If it's not observable, can it be totally controlled? (no)
  • Fault Tolerance: replication and isolation.
  • Responsibility (and understanding the big picture) leads to desire for observability (and isolation/duplication). >> PLEX team at VP is a form of big picture.

What's Happening in Your Production Data and ML Systems  - Don Sawyer, PhData

  • Most practical of the lectures.
  • Focus on decoupled systems: Data warehouse, ML Models.
  • Talked Provenance as both origin and change over time.
  • Timestamp everything UTC (use Google Time API as an example to change it during compute).
  • Focus on: audit trails, data quality, repeatability, added info (pipeline).
  • Metadata payload.  PROCESS: id/version, start/end, transformations, inputs, configuraitons, DATA VERSIONS: traces of issues, data change history, defect data, LINEAGE: sources, frequencuu of read.
  • Last point was a little messy (from me) but you want to trace right down to the node data touched in transit so you can hydrate anything from the last known good state.
  • NOT ALL DATA RECORDS require granular povenance.  Can be expensive (so much data).  Use a flexible or generic schema.  Don't use S3 (slow).  Storage considerations.
  • Storage: 1.) attach info to the record (can get big, note that Avro and Parquet are meant to do this), 2 send a separate event message - separate provenance API, 3.) only track some.  Note that for API approaches you may end up going down a rabbit hole of tracking the tracking api.
  • Alternatives: Amundsen (Lyft), Marques (WeWork), DataBook (Uber), DataHub (LinkedIn)
  • Look at Apache Nifi (there's a pluralsight class)

Evolving Chaos Engineering - Casey Rosenthal, Verica

  • Ships, shoes, fruit (apricots), helium mining.  He's a very funny guy.
  • LOOK FOR  A VIDEO to watch with the team:  https://www.youtube.com/watch?v=JfT9UxcEcOE
  • Principlesofchaos.org
  • Reversibility: blue/green, feature flags, ci/cd, agile to waterfall.
  • Moved responsibility away from the people who do the work (hierarchy)
  • Myths:
  • 1. remove the people causing the accidents.
  • 2. document best practices and use runbooks.  (most interesting problems are unique)
  • 3. defend against prior root causes, aka defense in depth.  Root cause analysis: "at best, you are wasting your time."  Was our sponsor audience issue an example?  The answer was in part to restrict audience size.  But the dig highlighted system no longer supports system-wide features after growth, high processing cost of feature, inability to test with all users, etc.
  • 4. enforce procedures
  • 5. avoid risk
  • 6. simplify
  • 7. add redundancy
  • Do NOT eliminate complexity.  Navigate it.  CI, CD, CV - continuous verification (here's a link to a CV article:  https://thenewstack.io/continuous-verification-the-missing-link-to-fully-automate-your-pipeline/).  That's New Relic for us.
  • Has two books: Chaos Engineering and Learning Chaos Engineering.  First book comes out June 2020.

Sunday, March 29, 2015

Catch Up - Part IV - Code Freeze 2015

Code Freeze had a security focus this year.  And although much of it had to do with things, the lessons were still good.  I know I've been talking about Stephen Checkoway's presentation on how to hack a car to my team for months.  He and Bruce Shneier both had great presentations.  The break outs about bitcoin and secure email were a little less inspiring, but gave me time to go have lunch with Ming at the local Asian restaurant.  Checkoway's takeaway is that cars were built with the expectation that as long as you were within the firewall/existing system you were safe.  That led to buses being connected to the extent that if you could hack the radio or force a buffer overflow via a CD or get a car outside the 4G network so that it had to communicate via traditional phone, you could hack everything right up to the point of driving it remotely (horn, shut it down, listen if there's a phone, unlock the doors, start it, etc).  It's a good lesson if you trust your database to serve up clean data all the time just because it's your database.



The panel at the end was good and elucidated on some of what Shneier was discussing earlier in the conference.  Shneier doesn’t like the worst case approach in security.  There’s too much of that in his opinion (operating in the moment), although he recognizes it’s somewhat at odds with his message around considering your adversary in advance, and it leads to “muddling through.”  That’s an old rules approach and “the old rules don’t apply.”  Checkaway agrees and sees it as positive that there’s been a shift from something might happen, to “something will happen and we have to deal with it.”

Bruce Shneier noted that getting companies to share their security data is valuable, but not easy (companies worry about their reputation).  Sharing virus information and other information helps many other industries and initiatives.  I include a few of my notes below.  Mostly for me:

  • Systems that support people instead of replacing people involve reduced switching costs (example, the military).
  • Speed is the advantage in an iterative loop of the hacker versus the hacked.
    • Observe, orient, decide, act (OODA)
    • Detection = IT.  Response = People.
    • So improve them, training them, use the OODA loop, consider the switching cost.
  1. We are losing control of our IT infrastructure.”  This means we’re also losing visibility.  We are also using devices we have less visibility/control over.
  2. Attacks are more sophisticated in their skill and focus.  There is relative security and absolute security.  “A sufficiently motivated attack will get in.  Full stop.”  The attacker has the advantage.
  3. There is an increase in government involvement in cyberspace, pro and con.  Some countries are selling cyberweapons (as arms dealers; turnkey solutions).
  • The economy affects security:
    •  Switching costs (WP vs. Word)
    •  Managerial costs
    •  Fixed costs (stamping out more copies)
    •  Lemons Market (cheap and easy wins and that’s not always good)
    •  Risk seeking when it comes to losses, risk adverse when it comes to gains.
    • “People will drive to work and fear terrorism.  What are you thinking?”
    •  Security is hard to sell.  Management is willing to “take the chance.”  Think burglar alarms  Car alarms
  •  Look up Asian Disease Experiment on Wikipedia: http://en.wikipedia.org/wiki/Framing_effect_%28psychology%29
  •  Look up his Crypto-gram newsletter.
  • There’s an MSST degree in security at the U of MN.