Why Privacy Matters for AI
AI systems often process large amounts of personal information—customer data, employee records, communications, and more. This creates specific privacy considerations:
Data collection
AI may collect more data than necessary if not carefully designed
Processing transparency
AI decision-making can be opaque to individuals affected
Third-party services
AI often involves sending data to external providers
Data retention
AI systems may store data longer than needed for training
Getting privacy right isn't just about compliance—it builds trust with customers and protects your business from regulatory and reputational risks.
NZ Privacy Act 2020 Overview
The Privacy Act 2020 governs how agencies (including businesses) collect, use, store, and disclose personal information. Key principles relevant to AI:
Purpose of collection
Requirement: Only collect personal information for a lawful purpose connected to your function or activity.
AI implication: Don't collect more data than your AI actually needs.
Source of information
Requirement: Generally collect directly from the individual concerned.
AI implication: Be careful with AI that scrapes or infers data from other sources.
Collection from the individual
Requirement: When collecting, tell individuals what data you're collecting, why, and who will receive it.
AI implication: Your privacy policy must cover AI processing clearly.
Storage and security
Requirement: Protect personal information against loss, misuse, and unauthorised access.
AI implication: Secure AI systems and any third-party AI services you use.
Use of information
Requirement: Only use information for the purpose you collected it for.
AI implication: Using customer data to train AI may require explicit consent.
Disclosure
Requirement: Only disclose to authorised parties for authorised purposes.
AI implication: Sending data to AI providers (like OpenAI) is disclosure.
AI-Specific Considerations
AI introduces unique privacy challenges beyond traditional data processing:
Third-Party AI Services
Using services like OpenAI, Claude, or Microsoft Copilot means sending data overseas. Consider:
- • Review their data processing agreements and privacy policies
- • Understand where data is stored and processed
- • Check if they use your data for model training (and opt out if possible)
- • Ensure contracts cover data protection requirements
Automated Decision-Making
When AI makes decisions affecting individuals (hiring, credit, customer service):
- • Ensure humans can review and override AI decisions
- • Be prepared to explain how decisions were made
- • Test for bias and discrimination
- • Allow individuals to challenge automated decisions
AI Training Data
If you fine-tune AI models or build custom solutions with customer data:
- • Training is a form of "use"—ensure you have appropriate consent
- • Consider anonymisation or synthetic data alternatives
- • Document what data was used and how
- • Respect deletion requests (right to be forgotten)
Data Handling Principles
Follow these principles when implementing AI systems:
Data Minimisation
Only collect and process data that's actually necessary for your AI to function.
Example: If your chatbot only needs to know customer name and query, don't send their full account history.
Purpose Limitation
Use data only for the specific purpose you collected it for.
Example: Support chat data collected for customer service shouldn't be used for marketing without consent.
Transparency
Tell people clearly how AI is used in your business.
Example: Update your privacy policy to explain AI processing and inform users when they're interacting with AI.
Security
Protect data at rest and in transit with appropriate technical measures.
Example: Use encryption, access controls, and secure API connections to AI services.
Retention Limits
Don't keep data longer than necessary.
Example: Set up automated deletion of chat logs after a defined period.
Access & Correction
Enable individuals to access their data and request corrections.
Example: Have a process for handling data access requests that includes AI-processed data.
Compliance Checklist
Use this checklist when implementing AI systems:
Before Implementation
During Operation
Ongoing
Responsible AI Practices
Beyond legal compliance, responsible AI practices build trust and reduce risk:
Human oversight
Maintain human review for significant decisions. AI should augment, not replace, human judgment for consequential choices.
Bias testing
Regularly test AI outputs for bias across different demographic groups. Address any disparities found.
Explainability
Be able to explain, in plain language, how AI systems make decisions that affect customers.
Feedback mechanisms
Provide ways for users to report issues with AI outputs and actually act on that feedback.
Continuous monitoring
Monitor AI performance over time. Models can drift, and what worked yesterday may not work tomorrow.
Practical Implementation
Here's how to put these principles into practice:
For AI Chatbots
- • Display "You're chatting with an AI assistant" clearly
- • Don't store chat logs longer than necessary (e.g., 90 days)
- • Offer easy escalation to humans for sensitive topics
- • Review a sample of conversations regularly for issues
For Workflow Automation
- • Only pass necessary data fields to AI processing steps
- • Use NZ-based or approved overseas data processing where possible
- • Log what data flows through automated workflows
- • Review integrations when providers update their terms
For Document Processing AI
- • Review what data is sent to AI for processing
- • Consider redacting sensitive fields before AI processing
- • Ensure processed documents are stored securely
- • Have clear retention and deletion policies
Getting Help
Privacy compliance for AI can be complex. Here are your options:
Office of the Privacy Commissioner
Free guidance and resources for NZ businesses on Privacy Act compliance.
privacy.org.nz →Privacy Professionals
For complex AI implementations, consider engaging a privacy consultant or lawyer with AI experience to review your approach.
AI Implementation Partners
Work with AI agencies (like us) that build privacy considerations into their implementation process from the start.
Need privacy-conscious AI implementation?
We build AI solutions with privacy by design. Talk to us about implementing AI that respects customer data and complies with NZ requirements.