Seven engines. One workspace.
A secure Flask backend paired with a fully responsive, framework-free JavaScript frontend delivers a desktop-grade data-preparation experience directly in the browser.
Backend Server
Flask application handling routing, session security, permission enforcement, and the CSV upload endpoint.
Data Parsing Layer
Delimiter sniffing, 19-class type inference, and index-mapped row formatting for compact network payloads.
Cleaning Engine
Whitespace, casing, fill/blank, fuzzy clustering, sandboxed expression transforms, and Unicode/Arabic/date/number/currency normalization.
Schema Engine
Split, join, derive, rename, remove, and move column operations — with facets and rules cleaned up automatically.
Facet Engine
Numeric bins, box-plot statistics, scatterplot coordinates, duplicate detection, and text filtering — computed server-side.
Database Layer
Parameterized queries with automatic MySQL-to-SQLite fallback and schema self-upgrade on startup.
Frontend Controller
A 5,500+ line asynchronous single-page state machine covering the grid, facets, validation, statistics, DQ scoring, history, and administration UI.
19-class automatic type inference
Uploads are sniffed for delimiter, stripped of blank rows, and converted to a compact index-mapped structure. Then every column's first 100 non-empty values run through a cascade of pattern detectors.
A grid built for real analysis sessions
A paginated grid — 100 rows per page — with sticky headers, drag-resizable columns, and per-cell status highlighting, surrounded by tool panels you can rearrange to fit your screen.
Column context menu
Every header exposes a single dropdown with every relevant facet, cleaning, schema, and validation action for that column's detected type.
Multi-select & bulk actions
Select several columns and a floating action bar appears — validate or facet them together in one step.
Row status indicators
Live status pills — All good, Worth a look, Needs fixing, or Pending — reflecting each row's worst active validation outcome.
Responsive side panels
Collapsible, drag-resizable panels (220–700px) for Facets, Rules, History, Export, Stats, and DQ Dimensions — with an off-canvas drawer below 860px.
Layouts & themes
Both Bars, Left Only, or Right Only layouts, plus light and dark themes persisted via localStorage.
Toast notifications
A bottom-center toast confirms success, surfaces errors, and explains permission-denied actions throughout the app.
An instant statistical lens on any column
Facets are computed server-side for performance and rendered as interactive panels — each with its own PDF export.
Text facet
Unique values with counts and frequencies, sortable by name or count — click a term to filter the grid.
Numeric facet
Groups values into 12 equal-width histogram bins to reveal distribution and skew.
Box plot facet
Tukey five-number summary with 1.5× IQR whiskers, outlier detection, and a one-click "Filter outliers" control.
Scatterplot facet
Plots paired values between two numeric columns, aggregating repeated coordinates into density counts.
Duplicates facet
Surfaces every value occurring more than once, sorted by descending frequency.
Text filter facet
Substring or regular-expression matching with case-sensitivity and invert-match toggles.
From trimmed whitespace to repaired mojibake
Two tiers of column-wide operations — applied instantly, automatically re-validated, and recorded in the undo/redo history.
Standard operations
- ✓Trim / collapse whitespaceRemove leading/trailing spaces, or merge repeated inner spaces into one.
- ✓Case conversionConvert a column to Title Case, UPPERCASE, or lowercase.
- ✓Fill down / blank downPropagate the last non-blank value forward, or clear consecutive duplicates.
- ✓Cluster & edit (fuzzy merge)Group near-duplicates via Key Collision (Fingerprint) or Nearest-Neighbor (Levenshtein) clustering, then merge to one canonical value.
- ✓Transform by expressionSandboxed Python expression per row — referencing value and row[...] — with a live 10-row preview and inline error reporting.
Advanced & regional operations
- ◆Unicode normalization (NFKC)Collapse visually-identical compatibility characters into one canonical form.
- ◆Accent / diacritic removalStrip accents via NFD decomposition — é becomes plain e.
- ◆Arabic text normalizationRemove Tashkeel diacritics and normalize Alef, Teh Marbuta, Yeh, and Tatweel variants.
- ◆Emoji / invisible / control removalStrip emoji glyphs, zero-width characters, and non-printable control characters.
- ◆Encoding repairDetect and repair common Latin-1/CP1252 mojibake via re-encode/decode round-trips.
- ◆Date / number / currency normalizationParse 13+ date formats into ISO 8601; convert Arabic/Persian digits and mixed separators; strip currency symbols into clean numerics.
Reshape columns without leaving the grid
Split into columns
Split values into adjacent columns via a string or regex delimiter, with an optional limit and source-column removal.
Join columns
Combine multiple selected columns with a chosen delimiter into one new, named column.
Add derived column
Compute a new column from a Python expression evaluated against other cells in each row.
Rename / remove / move
Relabel, delete, or reposition columns — associated facets and validation rules are cleaned up automatically.
Manual type override
Reclassify a column's detected type from the context menu, instantly refreshing its default rules and reports.
23 rule classes. Three severities. One policy.
Attach rules to one or many columns at once. Export rule sets as a CSV template and re-import them in bulk to replicate a data-quality policy across similar datasets.
| Rule | Class | Checks |
|---|---|---|
| Not empty | Core | Flags blank / null / whitespace-only cells |
| Pattern (regex) | Core | Match against a custom regular expression |
| Number range | Core | Numeric value within a min–max range |
| Allowed values | Core | Value is in a semicolon-separated whitelist |
| Unique (no duplicates) | Core | No repeated values in the column |
| Expression (GREL) | Core | Custom boolean expression per row |
| Valid Email / URL / Phone | Format | Standard format syntax checks |
| Valid National ID / Iqama / Mobile | Regional (KSA) | Structural checks for identifiers and numbers |
| Valid Latitude / Longitude | Geospatial | Numeric within valid geographic bounds |
| Valid Date / Timestamp | Temporal | Parses as a real calendar date or date-time |
| Valid Currency / Boolean / JSON / UUID | Format | Structural / semantic format checks |
| Valid IP / MAC / Postal Code | Network & Format | Structural pattern checks |
Instant grid highlighting
Cell backgrounds turn red, orange, or blue with hover tooltips explaining exactly why a value failed.
Validation reports sidebar
Aggregates issue counts per column and per rule, expandable down to the specific failing row indices.
Rule templates
Export a rule set as CSV and re-import it in bulk — one quality policy, many datasets.
The 10-dimension Data Quality Index
A defensible, quantified quality score — the equal-weighted average of ten dimensions, rendered as color-coded progress bars and exportable as CSV or a formatted PDF scorecard.
Descriptive statistics — quantitative columns
Descriptive statistics — qualitative columns
A separate Statistical Functions panel rolls a subset into a weighted Quality Score — Completeness 30% + Consistency 30% + Accuracy 40% — for a fast per-column health check.
Ten permission flags, enforced on every route
Session-based authentication with PBKDF2-SHA256 password hashing and HTTP-only, SameSite cookies. Every sensitive server route is wrapped in a permission decorator, and superadmins manage access from an in-app console — no direct database access required.
| Permission | Grants access to |
|---|---|
| can_upload | Uploading new CSV files |
| can_facet | Building and viewing facets / filters |
| can_edit_cell | Standard cell cleaning operations |
| can_edit_column | Split, join, rename, remove, move, derive columns |
| can_advanced_clean | Unicode / Arabic / encoding / date / number / currency normalization |
| can_validate | Creating and running validation rules |
| can_view_stats | Viewing the Descriptive Statistics panel |
| can_view_dq | Viewing the Data Quality Dimensions panel |
| can_export_report | Exporting reports and datasets |
| is_superadmin | Bypasses all checks; exclusive User Management access |
Everything exports. Everything rewinds.
Multi-format data export
CSV, TSV, or JSON — scoped to the full file, valid rows only, errors only, or warnings only.
Validation report export
A shareable CSV or print-ready PDF of validation results for audit or handoff.
Statistics & DQ export
Both analytical panels export independently to CSV or a styled PDF scorecard.
Facet chart export
Print the active histogram, box plot, or scatterplot to a PDF-ready layout.
Snapshot timeline
Every mutating action captures a full dataset snapshot. Click any History entry to jump directly to that exact point — not a linear undo stack.
Experiment freely
Aggressive transforms are safe when any state is one click away. That confidence is the point of the timeline.
Boring where it should be, fast where it counts
| Layer | Technology | Role |
|---|---|---|
| Backend | Flask 3.0 (Python) | Routing, sessions, REST-style JSON APIs |
| Database (primary) | MySQL via PyMySQL | Production multi-user persistence |
| Database (fallback) | SQLite 3 | Automatic offline / dev fallback |
| Security | Werkzeug Security · python-dotenv · cryptography | Password hashing, secrets, session cookies |
| Frontend | Vanilla JavaScript (ES6+) | Single-page state machine, no framework |
| Styling | CSS custom properties · Plus Jakarta Sans | Light/dark theming, responsive layout |
| Templating | Jinja2 (Flask render_template) | Server-rendered shell pages |
Known gaps, tracked openly
Identified during the 2.0 review and tracked on the product roadmap — because a data-accuracy platform should be accurate about itself.
The team behind Misbar
Misbar is developed by DataMisbar in Jeddah, Saudi Arabia — pairing hands-on data engineering with senior statistical and analytical expertise spanning academia, government, and industry.
Dr. Ateq Ahmed AlGhamedi
Brings decades of statistical science and executive leadership to Misbar's quality methodology — from data-driven organizational transformation to national-level advisory work.
- Former Chief Executive Officer, Wadi Jeddah Company
- Advisor to the Minister of Labor & Social Development
- Vice Dean, King Abdulaziz University
- Ph.D. in Statistics, Colorado State University (USA); Executive Training, INSEAD (France)
Dr. Nazmul Ahasan Maruf
Designs and builds the Misbar platform end to end — the parsing, cleaning, validation, and data-quality engines — drawing on research in machine learning and medical image analysis.
- Ph.D. research in radiomics & deep learning for cancer detection
- Full-stack engineering: Python, Flask, React, and data pipelines
- Statistical data analysis & weather-data management at DataMisbar
Put your data under the probe.
Misbar 2.0 is internal product documentation made real — upload a CSV, watch 19 detectors classify it, and walk away with a defensible quality scorecard.
Request access