Data Migration Framework
Prorigo’s Data Migration Framework can migrate data and files from any source system to target system. Data Migration Framework uses ETL based tools to extract, transform, and load data and files. Supports migration for both On-Premises and SAAS environments. DMF also includes validation tool to validate migrated data.
Benefits of using Data Migration Framework
ETL Methodology for Speed and Scalability
A framework based on ETL methodology that offers a collection of fast, scalable, and configurable tools to streamline data migration.
Configuration-Driven Approach
Ready-to-use tools tailored for various established PLM applications, enabling quick and efficient setup.
Comprehensive Metadata and File Support
Handles all types of files and metadata, including historical records and revisions, ensuring seamless compatibility.
High-Performance Design
Incorporates unique techniques like direct database access and multiple parallel threads to significantly enhance performance.
Large-Scale Data Handling
Capable of migrating datasets with over 100 million records in just 16 hours, making it ideal for complex projects.
End-to-End Migration Solution
Supports the extraction and loading of full or incremental datasets with the flexibility to specify date and time ranges.
Real-Time Progress Tracking
Provides detailed and real-time diagnostic information, allowing for continuous monitoring of activities during the migration process.
Automated & Scheduled Migrations
Facilitates automation through chained tools for delta migrations, allowing scheduled data updates at regular intervals.
Reliable and Proven Results
Tested extensively across complex, large-volume projects to ensure accurate, reliable data migration with faster validations and reduced manual errors.
Detailed Dataflow of Data Migration Framework
Feature
Data Extractor
- One-Command Extraction: Extract metadata and files seamlessly for a pre-defined process area spanning multiple types.
- Config-Driven Process: Use a configuration file to define types, relationships, and file formats for precise extraction.
- Comprehensive Data Support: Extract object data, relationship data, files, and history effortlessly.
- Flexible Extraction Modes: Perform full or incremental extractions based on a specified date range.
- Optimized Performance: Leverage parallel extraction to significantly reduce processing time
- Filtered Extraction: Extract data based on vault, policy, state, date range, and other custom filters.
- Direct Database Access: Bypasses APIs for a faster, more efficient extraction process.
- Comprehensive Diagnostics: Provides detailed status updates and record counts for full visibility.
- Enterprise-Scale Performance: Capable of extracting up to 100 million records in just 8 hours.
- Sample Extractor Command -
ProrigoEnoviaOracleDataExtractor --fromDate "2022-05-08 14:40:52" --types "ManufacturingPart, Part, Document"
Feature
Data Transformer
- Automated Data Processing: Converts extracted source data into a structured format for easy loading into the target system.
- Intelligent Transformation: Supports CAD model transformations, including thumbnails, derived outputs, viewables, subcomponent rollups, and classification.
- Optimized Performance: Enables parallel processing for faster execution while ensuring data integrity through validation and attribute massaging.
- Configurable Workflow: Driven by customizable configurations to define transformation sequences, parameters, and output structure.
- Smart Relationship Handling: Splits, merges, and maps items and relationships efficiently from the source system.
- Comprehensive Diagnostics: Provides real-time insights into transformation status, counts, and potential errors.
- Sample Transformer Command -
ProrigoEnoviaToArasTransformer --transformType "CadMajorMinor, CadRollup"
Feature
Data Loader
- One-Command Data Load: Import transformed files into the target database with a single command.
- Config-Driven Execution: Uses a configuration file to define types, relationships, and file formats for structured loading.
- Comprehensive Data Support: Loads object data, relationships, files, revisions, and history seamlessly.
- SSIS-Powered Processing: Utilizes SQL Server Integration Services (SSIS) for efficient large-scale data movement.
- Dynamic SSIS Package Handling: Supports both prebuilt SSIS packages and on-the-fly package generation based on configuration.
- Dependency Management: Automatically sequences loads to maintain data integrity across types and relationships.
- Vault Integration: Directly copies files to InnovatorVault, creating the necessary folder structure.
- Automated Dependency Resolution: SSIS package logic ensures data dependencies are resolved dynamically (e.g., looking up users to set creator properties on parts).
- Seamless SQL Server Connectivity: Leverages SSIS for fast, reliable, and scalable data transfers.
- Incremental Updates: Supports updating existing data without requiring a full reload.
- Parallel Processing: Enables simultaneous data loads for multiple types to enhance efficiency.
- Enterprise-Scale Performance: Capable of loading up to 100 million objects in just 8 hours.
- Sample Loader Command -
ProrigoSsisDataLoader --types "ManufacturingPart, Part, Document"
Feature
Data Validator
- One-Command Validation: Ensures data accuracy after loading with a single command.
- Config-Driven Validation Rules: Uses a configuration file to define validation rules for all types and relationships.
- Comprehensive Data Checks: Validates both object and relationship data for consistency.
- Predefined & Custom Rules: Includes built-in validation rules and supports custom rule creation based on business needs.
- Dependency Handling: Automatically sequences validations to maintain data integrity across types and relationships.
- Mandatory Field Checks: Ensures required fields are populated and default values are correctly assigned.
- Customizable & Extensible: Allows businesses to tailor validators to meet specific requirements.
Ready to build your next idea?
Reach out to us!
Join the Prorigo family and collaborate with a trusted partner to bring your ideas to life. With a proven track record and a growing list of satisfied clients, we're ready to support your next big project. Connect with us today!