Tapan Parmar👋
Versatile Full Stack Developer with expertise in Python backend development, frontend technologies, and cloud services
Services:
Python, Django3, Shell scripting , AWS Batch, EC2, Fargate Spot instances, ECR, Eventbridge, Cloudwatch, Lambda Sentry, Amplitude PostgreSQL, DynamoDB, ElasticSearch , Nginx, Docker
https://www.sndlegal.com/Overview
Snd Legal is one of a kind product which is made from expertise of lawyers, Legal firm representatives, Engineers and business innovators. Simply using pre-built integrations for data transfer, On-demand scanning capabilities customer can make sure that all their businesses always stay connected and all compliances are measured with secure access, and software supports any frequency and amount of scans necessary which works for small startups to large corporations
Wireframing and Prototyping: Developed an end-to-end technological solution to help companies navigate compliance obstacles with ease. Led the entire project and created the necessary infrastructure using serverless and server-based stacks on AWS technologies such as AWS Batch, AWS Lambda, AWS Eventbridge, AWS Cloudwatch, AWS ECR, and AWS EC2. Built a Django-based dashboard website and custom automation scripting solution using Python, enabling companies to focus on their business and clients without worrying about industry regulations. Developed custom syncing logic to transfer data, files, and signals between multiple integrations, including building SFTP server-based file syncing, AWS S3-based file syncing, and database transfer. Designed a dashboard within the Django framework that provided real-time insights to companies on how much data they were transferring, statistics about processed data, and summaries for specific types of data such as Emails, Mobile numbers, and Pincodes. Incorporated indications for any anomalies detected during scans, along with correct file details at the line level, to enable quick and efficient issue resolution. Created deployment and git branching strategies to add new job types and maintain existing automation scripts effortlessly. Developed a dashboard for super admins and company admins to use the platform with varying levels of access and insights over company vs organization levels.
Challenges
Building Cost:
- Challenge: Building cost effective Data transfer pipelines using Serverless + AWS components that can provide multifold scaling on demand basis
Maintaining accuracy:
- Challenge: Maintaining accuracy (upto 99.5%) while data monitoring to report data anomalies and report it based on scan types
Architecture:
- Challenge: Building code architecture in a way that works with multiple filetypes and server types, with central logging and monitoring systems as well as common scanning procedures
Results/Conclusion:
In the development phase alone, we tried transferring several million files, tested the platform's accuracy, and achieved 100% accuracy in all levels of matrices, with O failures in transferring files and optimal cost at infrastructure. The platform could handle large scale at any point of time with 99.99% of uptime due to the serverless components, and Django was working with the Eventbridge events it receives. Also, to ensure the reports are accurate, we managed separate messaging queues with DL queuing to ensure parity is always there. Customer was able to transfer all manual processes related to file transfer and compliance measurement within his own organization as well as acquire multiple end clients
