I’m testing AI automation for my e-commerce clients – here’s my Stage 1 approach and why I’m verifying everything manually first
The Problem
One of my e-commerce clients (let’s call them Client X) sells products online and works with multiple suppliers. Every time a customer places an order, the process looks like this:
- Customer orders on their website
- Client X forwards the order to their supplier
- Supplier ships the product and emails tracking information
- Client X needs to extract the tracking number from the supplier email
- Client X updates the customer with tracking information
Except Client X processes dozens of orders per week. That’s dozens of supplier emails to open, read, find the tracking number, copy it, and update the customer.
Time spent: 5-10 hours per month just on tracking number extraction and customer updates.
The AI Opportunity
Could AI do this automatically?
In theory, AI should be able to:
- Read the supplier email
- Identify the tracking number
- Extract it accurately
- Format it correctly
- Trigger a customer update
If this works, Client X saves 5-10 hours per month. Multiply that across multiple clients, and we’re talking about significant time savings.
But can AI do this RELIABLY?
Why I’m Not Just “Turning It On”
Here’s what I could do:
- Find an AI tool that claims to extract tracking numbers
- Connect it to Client X’s email
- Let it run automatically
- Hope it works
This is a terrible idea.
Why? Because if AI gets it wrong:
- Customers get incorrect tracking numbers
- They can’t track their orders
- They contact Client X asking where their package is
- Client X looks unprofessional
- Customer trust is damaged
- Potential refund requests or complaints
Bad automation is worse than no automation.
My Stage 1 Approach: Test and Verify
Instead, I’m taking a careful, staged approach:
Stage 1: Manual Verification (Current Stage)
How it works:
- Client X forwards me supplier emails (or I access them with permission)
- I use AI to extract the tracking number
- I manually verify every single extraction against the original email
- I document: Did AI get it right? Did it get it wrong? What kind of errors occurred?
- I track accuracy rate over 20-30 emails
What I’m measuring:
- Accuracy rate (what percentage does AI get exactly right?)
- Error patterns (does it struggle with certain supplier formats?)
- Edge cases (what happens with unusual emails?)
- Consistency (does accuracy improve or vary?)
What happens next:
- If accuracy is 95%+: Move to Stage 2
- If accuracy is 85-95%: Refine the AI prompts and test again
- If accuracy is below 85%: This approach doesn’t work, try different method
Stage 2: Semi-Automated with Spot Checks (Future)
Once I prove Stage 1 works reliably:
- AI extracts tracking numbers automatically
- Client X spot-checks 10-20% of extractions
- We monitor for any customer complaints about tracking
- We continue to track accuracy
Stage 3: Fully Automated (Future)
Only after Stage 2 proves reliable over 2-3 months:
- Full automation with monitoring systems
- Fallback processes if AI fails
- Regular accuracy audits
What I’m Learning So Far
I’m currently in Stage 1, and here’s what I’m discovering:
What AI does well:
- Extracting tracking numbers from standard formatted emails
- Identifying tracking numbers even when surrounded by other text
- Working with different supplier email formats
Where AI struggles:
- Emails with multiple tracking numbers (which one is correct?)
- Unusual formatting or non-standard layouts
- Emails in different languages
- Tracking numbers that look similar to other number sequences
The surprising part:
AI is more accurate than I expected for standard cases, but it makes confident mistakes on edge cases. Without manual verification, I wouldn’t catch these errors until a customer complained.
Why I’m Sharing This
I’m not an AI expert. I’m a digital marketing and SEO consultant who’s learning how to use AI tools safely and effectively for my clients.
I’m sharing this experiment because:
- AI is changing how we work, and I want to stay ahead
- I believe in testing thoroughly before implementing
- I want to show the REAL process, not just the polished results
- Other small businesses are wondering the same things
What This Means for Your Business
If you’re thinking about using AI in your business:
- Don’t just “turn it on” – Test carefully first
- Verify outputs manually – Especially when it affects customers
- Document what works – And what doesn’t
- Start with low-risk tasks – Not customer-facing processes
- Build verification systems – Before you automate
AI has huge potential to save time and improve efficiency. But it needs to be implemented thoughtfully, tested thoroughly, and monitored carefully.
Want to discuss AI automation for your e-commerce business? I’m happy to share what I’m learning and explore whether similar approaches could work for you.
Carrie | Consilium Design Ltd | Digital Marketing & SEO Consultant
Currently testing: AI tracking extraction
