How I Built VoxCalc — An AI-Inspired Next-Gen Calculator with Flutter, Google ML Kit & Voice NLP

TL;DR: I built a cross-platform scientific calculator in Flutter that lets you speak math equations out loud, point your camera at handwritten formulas, and watch them get solved instantly. Here's exactly how I did it.

🎯 Why I Built This

As a B.Tech CSE student, I was tired of building the same CRUD apps for my portfolio.

Everyone has a to-do list. Everyone has a weather app. I wanted to build something that genuinely impressed technical interviewers and showed I could work with AI APIs, device hardware, and complex state management — all at once.

So I built VoxCalc — a mathematical computational suite that combines:

🎙️ Voice-to-Math: Speak "what is 15 percent of 340" and get the answer
📸 Camera OCR: Point at a handwritten equation → it solves it
📈 2D Graph Plotter: Visualize any algebraic function with multi-touch zoom
💱 Currency Converter: Live exchange rates via REST API

Here's what I learned building it from scratch.

🛠️ The Tech Stack

Framework:     Flutter 3.x (Dart)
Architecture:  Clean Architecture (Feature-First)
AI / ML:       Google ML Kit (Text Recognition v2)
Voice:         speech_to_text package (NLP)
State Mgmt:    Provider / Riverpod
APIs:          Open Exchange Rates REST API
Storage:       flutter_secure_storage
Build:         Gradle with R8/Proguard minification

🏗️ Architecture: Why Feature-First Clean Architecture?

Most Flutter tutorials show you lib/screens/ and lib/widgets/. That works for demos. It breaks for real apps.

I used Feature-First Clean Architecture, which separates every feature into its own isolated module:

lib/
├── core/
│   ├── theme/
│   ├── utils/
│   └── constants/
├── features/
│   ├── calculator/
│   │   ├── data/
│   │   ├── domain/
│   │   └── presentation/
│   ├── voice_input/
│   │   ├── data/
│   │   ├── domain/
│   │   └── presentation/
│   ├── ocr_scanner/
│   │   ├── data/
│   │   ├── domain/
│   │   └── presentation/
│   ├── graph_plotter/
│   └── currency_converter/
└── main.dart

Why this matters: Each feature can be developed, tested, and deployed independently. When I added the currency converter 2 weeks after the core calculator, I didn't touch a single existing file.

🎙️ Feature 1: Voice-to-Math NLP Engine

This was the most exciting feature to build. The goal: convert natural spoken language into a solvable math expression.

The Challenge

speech_to_text gives you raw text like:

"what is fifteen percent of three hundred and forty"

That's useless to a math parser. You need:

340 * 0.15

My Solution: A Custom NLP Preprocessor

I wrote a Dart utility class that maps English phrases to mathematical operators:

class MathNLPParser {
  static const Map<String, String> _wordMap = {
    'plus': '+',
    'minus': '-',
    'times': '*',
    'multiplied by': '*',
    'divided by': '/',
    'percent of': '* 0.01 *',
    'squared': '^ 2',
    'square root of': 'sqrt(',
  };

  static String parse(String spokenText) {
    String expression = spokenText.toLowerCase();

    // Replace word numbers with digits
    expression = _replaceWordNumbers(expression);

    // Replace operator phrases
    _wordMap.forEach((word, operator) {
      expression = expression.replaceAll(word, operator);
    });

    // Clean up and validate
    expression = expression.replaceAll(RegExp(r'[^0-9+\-*/.()^\s]'), '').trim();

    return expression;
  }
}

Then I pass this cleaned expression to a math evaluation engine to get the result.

The UX Touch

While the user is speaking, I render a live audio waveform animation using a custom painter that reads microphone amplitude values in real time. This makes the feature feel futuristic and gives users confidence the app is listening.

📸 Feature 2: Camera OCR Scanner (On-Device, 100% Offline)

Using Google ML Kit Text Recognition v2, I built a live camera scanner that detects handwritten or printed mathematical expressions and solves them instantly.

Setup

# pubspec.yaml
dependencies:
  google_mlkit_text_recognition: ^0.11.0
  camera: ^0.10.5

The Core Scanner Logic

class OcrScannerService {
  final textRecognizer = TextRecognizer(
    script: TextRecognitionScript.latin,
  );

  Future<String?> recognizeFromImage(InputImage image) async {
    try {
      final RecognizedText result = await textRecognizer.processImage(image);

      // Extract only math-relevant characters
      String raw = result.text;
      String cleaned = _extractMathExpression(raw);

      return cleaned.isNotEmpty ? cleaned : null;
    } catch (e) {
      debugPrint('OCR Error: $e');
      return null;
    }
  }

  String _extractMathExpression(String raw) {
    // Keep only digits, operators, and math symbols
    return raw
      .replaceAll('×', '*')
      .replaceAll('÷', '/')
      .replaceAll(RegExp(r'[^0-9+\-*/().=\s]'), '')
      .trim();
  }
}

Why On-Device Processing Matters

I specifically chose ML Kit over cloud OCR APIs (like Google Vision API) because:

✅ Zero latency — no network round trip
✅ Privacy — user's handwriting never leaves their device
✅ Works offline — functions in airplane mode
✅ Free — no API billing ever

📈 Feature 3: Real-Time 2D Algebraic Graph Plotter

This was the most technically complex feature. I built a custom CustomPainter that renders mathematical functions as smooth curves on a coordinate grid.

class GraphPainter extends CustomPainter {
  final String expression;
  final double scale;
  final Offset offset;

  GraphPainter({
    required this.expression,
    required this.scale,
    required this.offset,
  });

  @override
  void paint(Canvas canvas, Size size) {
    final center = Offset(size.width / 2 + offset.dx, 
                          size.height / 2 + offset.dy);

    // Draw coordinate axes
    _drawAxes(canvas, size, center);

    // Plot the function
    _plotFunction(canvas, size, center);
  }

  void _plotFunction(Canvas canvas, Size size, Offset center) {
    final paint = Paint()
      ..color = const Color(0xFF00E5FF)  // Neon cyan
      ..strokeWidth = 2.0
      ..style = PaintingStyle.stroke;

    final path = Path();
    bool firstPoint = true;

    // Sample function at every pixel along x-axis
    for (double px = 0; px < size.width; px++) {
      double x = (px - center.dx) / scale;
      double? y = _evaluateAt(expression, x);

      if (y == null || y.isNaN || y.isInfinite) {
        firstPoint = true;
        continue;
      }

      double py = center.dy - (y * scale);

      if (firstPoint) {
        path.moveTo(px, py);
        firstPoint = false;
      } else {
        path.lineTo(px, py);
      }
    }

    canvas.drawPath(path, paint);
  }
}

I also implemented multi-touch pinch-to-zoom and pan gestures using GestureDetector to give users a native-feeling interactive graph experience.

🔒 Security & Build Optimization

Secure Storage

For saving user preferences and API keys, I used flutter_secure_storage which encrypts data using platform-native keystore (Android Keystore / iOS Keychain):

final storage = const FlutterSecureStorage();

// Write
await storage.write(key: 'api_key', value: userApiKey);

// Read
String? key = await storage.read(key: 'api_key');

R8/Proguard Minification

In android/app/build.gradle, I enabled R8 code shrinking for the release build:

buildTypes {
    release {
        minifyEnabled true
        shrinkResources true
        proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'),
                      'proguard-rules.pro'
    }
}

This reduced the APK size by ~40% and made reverse engineering significantly harder.

🎨 UI Design Philosophy

I went with a dark glassmorphism theme with neon cyan highlights (#00E5FF). The design principle was: "make it feel like something from 2030".

Key UI decisions:

120 FPS micro-animations on button press using AnimatedScale
Custom slide-out drawer for feature navigation (instead of a boring tab bar)
Tactile haptic feedback on every key press using HapticFeedback.lightImpact()
Futuristic monospace font (JetBrains Mono) for the result display

🚀 What I Learned

Clean Architecture pays off immediately — when I added the currency converter, it took 4 hours, not 4 days
On-device ML is the future — no latency, no cost, better privacy
Custom painters are powerful — Flutter's canvas API can render anything you imagine
NLP is hard but solvable — even simple regex + word maps get you 80% of the way there
Security is not optional — even a calculator app handles user data that deserves encryption

📦 Check It Out

🌐 Portfolio: md-rounaq-ali.netlify.app
💻 GitHub: github.com/md-rounaq-ali/VoxCalc
💼 LinkedIn: linkedin.com/in/md-rounaq-ali

💬 What's Next?

I'm currently exploring adding:

LaTeX rendering for displaying complex mathematical notation
Step-by-step equation solver showing all working steps
Wolfram Alpha API integration for symbolic math

If you found this useful, drop a ❤️ or share it! I'm a B.Tech CSE student from Hyderabad actively looking for Flutter internship opportunities — feel free to connect on LinkedIn!

推荐订阅源

DEV Community