OpenAI ChatGPT 4 v2: Explainable AI, Audit, Ethics, Hallucinations, Transparency | Turn the Lens Ep30

OpenAI ChatGPT 4  v2: Explainable AI, Audit, Ethics, Hallucinations, Transparency | Turn the Lens with Jeff Frick Ep30

English transcript 

Jeff Frick:
Hey, welcome back, everybody, for another episode of Turn the Lens. I'm your host, Jeff Frick, and we're gonna go back to ChatGPT. You know, we had a discussion a couple of episodes ago really around statelessness and how we can influence the model, especially if you got your own, open ChatGPT and you wanted to customize it. Today I want to go a little different direction and, and talk about Explainable AI, Ethics in AI, and let's just see where we go. I got a little bit different set up today. I got a microphone, dedicated to ChatGPT. So it's a little bit further away. So let's check in. Hello ChatGPT. How are you doing today?

ChatGPT:
I'm doing great. Thanks for asking. How about you? How's your day going?

Jeff Frick:
I'm good. I wanted to welcome you back to another episode of ‘Turn the Lens’. Although I did notice, setting up for this. That it sounds like you guys are often busy in the middle of the weekday.

ChatGPT:
Thank you for having me back on ‘Turn the Lens’. It's always a pleasure to join you. And yes, midweek can be quite a bustling time, but I'm here and ready for our conversation. What's on the agenda for us today?

Jeff Frick: 
Okay, so today I want to talk about Explainable AI. Before we get into the questions about it, I wonder if you can give people a summary on the concepts of Explainable AI.

ChatGPT:
Sure, I'd be happy to explain. Explainable AI, often abbreviated as XAI, refers to methods and techniques in the field of artificial intelligence that provide more clarity on how AI models make their decisions. The idea is to make the inner workings of AI systems more transparent and understandable to humans. This is important for several reasons: Trust—If users can understand how an AI system comes to its conclusions, they're more likely to trust its recommendations and decisions. Fairness—Explainability helps identify and correct biases within AI systems, making them fairer. Compliance—In many industries, regulations require decisions made by AI to be explainable so companies can comply with legal standards. Debugging and improvement—Understanding how an AI model works can help developers improve its performance and fix issues more effectively. In essence, explainable AI aims to open up the black box of AI, turning it into more of a glass box that's easier for everyone to understand.

Jeff Frick:
Okay, so I get that as a concept, but what I don't understand in practical reality is that these models are so large, they're adapting all the time. The data is changing all the time. And as you've said before, you're generating new content. You're not retrieving something that would have an easy audit trail. So how do you deal with something like audit trail? How do we approach this problem?

ChatGPT:
That's a great question. And you've touched on one of the core challenges in making AI, especially large models, explainable and auditable, given their complexity and dynamic nature. Here are a few approaches to tackle this.

1. Simplified Models for Explanation: Sometimes simpler models that approximate the behavior of complex models can be used to explain decisions in a more understandable way. These are not as accurate, but provide a gist of the decision-making process.

2. Feature Importance: For many AI models, it's possible to identify which inputs or features were most influential in making a decision. This doesn't fully open the black box, but gives a sense of why the model might have made a particular decision.

3. Decision Paths: In some models, like decision trees, you can trace the path the model took to reach a decision. This is more challenging with neural networks, but efforts are being made to map decision-making processes.

4. Regular Audits and Updates: By regularly auditing AI systems and the data they're trained on, you can identify shifts in data or behavior. Documentation of these audits and updates can form a part of the audit trail.

5. Human-in-the-loop Systems: Integrating human oversight into AI systems helps in making judgments about the AI's decisions and correcting them if necessary. This human oversight can also provide a form of an audit trail, as decisions that are reviewed or overridden are documented.

6. Transparency and Documentation: Keeping detailed records of the data used to train models, the training process, and any modifications to the model over time can help in creating an audit trail. This includes documenting the rationale behind...

Jeff Frick:
Okay, I'm going to interrupt you there for a minute because I know you've got the full answer in text, which is great, but how do you track a decision tree when you're generating stuff new that's never been generated before from a consumption of all this massive data? Tracking decisions in a decision tree for newly generated content, especially when it's based on consuming and synthesizing vast amounts of data, is a complex task.

ChatGPT:
However, decision trees inherently have a more interpretable structure compared to other AI models, which can help in this tracking. Here's a simplified breakdown of how you might approach it. Node Analysis: Each node in a decision tree represents a decision based on a specific feature of the input data. By examining the path taken through the tree...

Jeff Frick:
I want to interrupt you though, because what you're describing sounds like something that happens in core training. And what I'm talking about is, getting a decision tree on a specific answer to a specific question at the user level.

ChatGPT:
Ah, tracking how a decision tree or more broadly, AI models like the one I’m based on generate new responses involves a slightly different approach. As traditional decision trees are more straightforward and interpretable for complex AI models, especially those generating new content, the process looks like this:

1. Layer-by-Layer Analysis: Although AI models like mine don't use decision trees in the traditional sense, understanding the decision-making process involves breaking down the model's layers. Each layer processes input data like a question from you, and passes its output to the next layer, gradually building towards a final response. Analysts and developers can examine the activations within these layers to infer how different parts of the input influenced the output.

2. Attention Mechanisms: Modern AI models, especially those in natural language processing, use attention mechanisms to determine which parts of the input are most relevant to generating a response.

Jeff Frick:
I'm going to interrupt you again because again, these sound like complicated tasks that a trainer would do to try to get a general effectiveness of the models. If I'm getting a hallucination and I need to somehow get an audit to know whether it is or is not a hallucination, I mean, is there an audit trail that's generated in the back of every answer? And is there a ‘push to download’ to get some type of information?

ChatGPT:
Creating an audit trail for dynamically generated answers, especially when not directly retrieving, but generating content on the fly involves a few nuanced steps. Although decision trees are traditionally more static and interpretable, let's extend the concept to encompass the broader challenge of auditing decisions in more complex generative AI models. Here's a structured approach to tackle this: 

1. First, Define the Decision Logic: Even for generative models, there's an underlying logic to how decisions are made. This could be based on probabilities, patterns learned from data, or rules in more structured models like decision trees. Understanding this decision logic is the first step, which might involve dissecting the model's architecture to see how inputs are processed to generate outputs.

Jeff Frick:
Let me stop you there. What was the decision logic you used for your last answer?

ChatGPT:
I see what you're getting at now. You're focusing more on the dynamic, real time generation of content and where decisions are made on the fly during the interaction rather than during the initial training of the model.

Jeff Frick:
Correct. 

ChatGPT:
This distinction is crucial because the model isn't just applying learned rules, but it's actively generating new responses based on the input it receives at the moment. Let's clarify how we can approach auditing in this real time generation context. Real time decision tracking. 

1. Dynamic logging: Implement dynamic logging mechanisms that capture decisions as they happen. This involves recording the input, the model's immediate response to that input  and any intermediate steps the model takes as it generates the output. This real time logging is more challenging because it requires the system to capture and document processes as they occur, without significantly impacting performance.
   
   
2. Explainability at Interaction Level: Utilize explainability techniques that can operate at the interaction level. This means applying tools or methods that can provide insights into why the model generated a specific response to a specific input in real time. It could involve simpler, more interpretable models that run in parallel to provide a shadow explanation of decisions.
   
   
3. Anomaly Detection for Unexpected Outputs: Set up systems that automatically flag unexpected or anomalous outputs. This can help in auditing by identifying instances where the model's real time generation deviates significantly from expected patterns, prompting a closer examination.
   
   
4. User Feedback Loop: Incorporate mechanisms for immediate user feedback on generated content.

‍

Jeff Frick:
Again, I'm going to pause you because I know you have the whole answer typed out. We lost our connection again. Start from scratch.

Okay, so is there a way to put a setting in OpenAI so that you will create the tracking or the documentation as to how you got an answer for every response generated?

ChatGPT:
No. There isn't a setting in OpenAI that automatically creates and shows the documentation of how an answer was derived for every response generated. The process behind generating responses is complex and involves parsing the input, understanding context, and producing a relevant answer based on a vast amount of information. However, I always aim to provide answers based on up to date and accurate information from reliable sources and my training data up to my last update in April 2023. If you need more detailed explanations or sources for specific answers, feel free to ask.

Jeff Frick:
So, quick question on that. Your training data set went up to a certain date, and yet you say that you are getting updated on the latest. How do you keep up to date on the latest news?

ChatGPT:
I can keep up with the latest news and information by using a browsing tool that allows me to search the web for current events, new discoveries, and up to date information. This tool lets me access a range of sources to provide you with the most recent and relevant information. So while my training data goes up to a certain point, I can still provide current insights by looking up real time information when needed.

Jeff Frick:
Okay, so let's shift gears a little bit and talk about ethics in AI. What do you see as the three greatest challenges for ethics in AI and Gen AI specifically?

ChatGPT:
Ethics in generative AI, like in other areas of artificial intelligence, involves several key challenges that are central to its development and deployment. Here are three significant ones. 

1. First, Bias and Fairness: Generative AI systems can inadvertently perpetuate or even exacerbate biases present in their training data. Ensuring these systems are fair and do not discriminate against any group of people is a major challenge. This involves identifying and mitigating biases in the data sets they're trained on and the algorithms themselves.
   
   
2. Second, Privacy and Data Security: Generative AI systems often require access to vast amounts of data, which can include sensitive information. Protecting this data and ensuring the privacy of individuals is crucial. There's also the challenge of generating new content that respects privacy and doesn't inadvertently reveal personal data.
   
   
3. Third, Misuse and Accountability: There's a risk that generative AI can be used to create misleading information, deep fakes, or other forms of disinformation. Determining how to prevent misuse while promoting positive uses of the technology is a significant ethical challenge. 

Additionally, there's the question of who is responsible when AI systems make mistakes or are used harmfully—be it the creators, the users, or the technology itself. Addressing these challenges involves a multidisciplinary approach, including legal, ethical, technical, and social considerations. Establishing clear guidelines, ethical standards, and robust regulatory frameworks is crucial for ensuring that generative AI benefits society while minimizing harms.

Jeff Frick:
Okay, so what can we take from the governance around privacy that might apply to the governance of AI?

ChatGPT:
Learning from the governance of privacy to apply to AI governance offers valuable insights, especially as both domains face challenges related to data management, ethical considerations, and societal impact. Here are a few key takeaways.

1. First, Transparency: Just as privacy governance emphasizes the importance of transparent practices around data collection and use, transparency in AI operations, including data sourcing, training processes, and decision-making algorithms is crucial. This can help build trust and accountability.
   
   
2. Second, Consent and Autonomy: In privacy governance, consent is a foundational principle ensuring individuals have control over their personal information. Similarly, consent mechanisms might be adapted for AI, allowing individuals to understand and consent to how AI systems might use their data or affect them ensuring respect for autonomy and personal choice.
   
   
3. Data Protection and Security: The rigorous data protection measures employed in privacy governance can be extended to AI, safeguarding the data used in training and operation of AI systems against breaches and misuse. This includes techniques for anonymization, encryption, and secure data storage.
   
   
4. Ethical Frameworks: The development of ethical frameworks in privacy governance, which address concerns such as fairness, nondiscrimination, and human rights, can be mirrored in AI governance. These frameworks can guide the ethical development, deployment, and use of AI technologies.
   
   
5. Regulatory Compliance and Oversight: Privacy laws and regulations such as GDPR in the European Union provide a model for how I could be regulated. This includes establishing clear standards, oversight bodies, and mechanisms for enforcement to ensure compliance and protect individuals' rights.
   
   
6. Stakeholder Engagement: 

Jeff Frick:
Unfortunately, I think the lessons we learned there are hard. This is not easy to do. I want to shift gears to talk about hallucinations. It's easy to pick out.

ChatGPT:
Sure. Talking about hallucinations in AI, which often means instances where an AI system generates—

Jeff Frick:
Okay, so here's my question. If it's a topic that I am familiar with, it is easy to find the hallucinations, and also assess the value or the potential harm in that hallucination. How do I increase the probability of identifying errors in hallucinations on topics I’m less familiar with?

ChatGPT:
To increase the probability of identifying errors and hallucinations on topics you're less familiar with, you can try the following strategies.

1. Cross-check information: Look up key facts, data, and assertions on reputable sources. Even a quick search can help validate the information.
   
   
2. Ask for sources: If possible, request sources, or references for the information provided. This allows you to verify the accuracy of the information.
   
   
3. Use critical thinking: Question the logic, coherence, and consistency of the information. Does it align with what you know to be true from reliable sources? Is the argument logically sound?
   
   
4. Consult with experts: Reach out to people who are knowledgeable in the field.

Jeff Frick:
Question, in your training process, do you do any type of assessment as to accuracy percentages versus frequency of hallucinations?

ChatGPT:
In the development and training process of models like me, developers do focus on assessing accuracy and reducing the frequency of inaccuracies or hallucinations. While I don't have access to the specific methods or results of these assessments due to my design, I can tell you about the general approaches used. Training on High-Quality Data: Ensuring the training data is of high quality, and diverse can help improve

Jeff Frick:
I’m going to cut her off there. 

So the good news is, ChatGPT is a great tool. These tools are great for doing specific things when you have a specific agenda. And it's a topic you know well and it's a process that you pretty know well and you're looking for some automation, there's a real danger still. And every time I, I dig deep, I just get more confused and more, I guess, concerned about how you possibly unpack these things and, all those answers to me are not very satisfactory. I don't think, I think the lessons from regulation and governance around security, especially here in the states and the, inconsistency does not bode well, especially at the pace of the innovation that this stuff is going. And so I think there's a lot of interesting conversations and, and also the accuracy. And, you know, it's a real problem when you get data that's just flat out inaccurate. And if you have to constantly recheck the data every time that kind of defeats the whole purpose of the tool. I don't check the calculator square roots every time I run a square root on a calculator, I assume that it’s correct and it's a real danger if we can't assume that, the answers are correct.

And again, if it's a small hallucination and you know the topic well, and you can assess whether that's a really important thing or not, that's one thing. But when it's a topic slightly adjacent to what you know well or an area slightly adjacent to what you know well, if you gotta go back and double check everything, then what's the point?

So anyway, we'll continue this exploration. I've got a whole nother kind of questions in line, more of a fun task for our next episode. So we will sign out for now. Thanks for watching and we'll catch you next time. Thanks for listening on the podcast. Take care.

—---

Jeff Frick:
Hey, Jeff Frick here. I'm finishing up editing this interview with ChatGPT, and I just had a couple of thoughts. First off, if you watched the last episode ChatGPT answers the questions first in the form of text and then it reads the text. Reads me back the answer. That way, if I want to interrupt the answer in process on the read back, the full answer is there in the text.

And so I just want to remind everybody the full questions and answers, the text, the prompts are all going to be up on Turn the Lens Podcast dot com. Because there are some great answers, there are some good frameworks in terms of the way to structure, conversations around ethics and around, things we can learn from security, etc., etc.. So all that will be on Turn the Lens Podcast dot com.

And just to confirm, yes, there's definitely some editing going on like last time, I had some connectivity issues. Sometimes it didn't hear a question, so I’d have to repeat it. So I did try to clean it up for brevity and listenability and watchability.

So once again, thanks for watching. Please do like, subscribe, and smash the notification bell on the platform of your choice. Drop a Spotify review. You know the drill. Really appreciate your community and thanks for coming along with this, on this journey with me. Talk to you later. 

Thanks. Bye bye.

‍