Ahmed Hasssan

Optimization of Prefill Time/ Time to First Token Generation of Mamba (SSM) for Faster On-device Inference

• Developed on-device LLM latency profiler for Mobile devices using Llamacpp plateform.
• Performed LLM latency profiling on CPU, GPU, and Samsung S23 Mobile CPU using Llamacpp to identify latency hotspots of different operations.
• Compared the throughput and latency of Transformer and Mamba models on CPU, GPU, and Samsung S23 Mobile CPU.
• Implemented divide and concur-based Parallel Scan on Pytorch and Llamacpp to reduce the on-device prefill/first token generation latency for Mamba-1.4 and Mamba-2.8 models with higher context length >5k.
• Reduced the prompt eval (TTFT) latency by Log(N) and achieved 1.39X speedup for Mamba 1.1B on a Samsung S23 device using Llamacpp.
• Working on Multi-threaded parallel scan to support the 5 threads of Samsung S23 Mobile device.
• Skills learned: LLM inference on mobile devices, edge computing, LLM profiling, latency optimization, multi-threading, and parallel computation.
• Tools learned: Termux for mobile devices, Llama.cpp and Linux.

Parametrized Effecient Fine-tuning of LLMs with SFT, DPO and ORPO on custom datasets

• Implemented instruction tuned SmolLM2-135M-Instruct model by fine-tuning on "ultrafeedback_binarized" dataset.
• Performed finetuning using SFT, DPO and ORPO to bechmark the performance of different apporaches.
• Prepared fine-tuning data based on prompt-content pairs formate.
• Findings: DPO includes perefrence alignment to imporve the LLM performance. SFT help LLM learn the domain specific knowledge.
• Skills learned: LLM inference instruct-tuning, Optimized fine-tuning.
• Tools learned: GPU and Linux.

RAG guided LLMs Inference on custom datasets

• Prepared RAG pipeline for LLM inference.
• Used Llama_index and "BAAI/bge-small-en-v1.5" embeddings to prepare the Vector Database for academic articles.
• Deployed Llama_index QueryEnginer to generate the Prompts through RAG retrieval.
• Imporved model performance through RAG guiding process.
• Skills learned: LLM inference on RAG, Prompt Engineering, RAG, Text Embedding.
• Tools learned: Llama_index.

Multimodal-3DGS: Audio-Visual 3D Rendering using 3D Gaussian Splatting. (Current Research Project)

• Implemented per pixel Gaussian learning for 3D scene rendering.
• Constrained Gaussian learning for resource optimal audio-visual rendering.
• Skills learned: 3D geometry, 3D spherical harmonics for color and sound, Gaussian intrinsics, 3D Gaussian pruning, and Transformer rendering.
• Tools learned: Cuda programming and GPU rendering.

Salient-Gaussians: 3D Gaussian Splatting with Direction Cosine and Gaussian Intrinsics based pruning.

• Implemented direction cosine for high quality 3D scene rendering.
• Implemented Gaussiang Intrinsics based pruning mask computation to reduce the active Gaussians for final rendering.
• Reduced the number of Gaussian by 5X without loosing the rendering quality in comprison to original 3DGS works.
• Reduced the memory utilization of overall rendering process by 1.7X.
• Skills learned: Direction cosine computation, 3D geometry, 3D spherical harmonics for color, Gaussian intrinsics, 3D Gaussian pruning, memory optimization, and efficient 3D scene rendering.
• Tools learned: Cuda programming and GPU rendering.

Low-precision and Memory Efficient Neural Radiance Fields (NeRF) with Hardware Accelerator Design

• Developed efficient NeRF with low-precision 3D Gaussian-based conical frustums for high-quality 3D scene generation.
• Implemented fixed-grid-based quantization scheme for low-precision representation of 3D Gaussian (8-nit and 3-bit).
• Designed and implemented statistically-aware boundary clipping and quantization for low-precision representation MLP activations (8-bit precison).
• Implemented adaptively rounding quantization scheme for low-precison weights representation to make the complete NeRF flow in 8-bit precision.
• Replaced the compute inefficient high-precision positional encoding with the fixed-precision (8-bit and 9-bit) look-up tables of Mean and Covariance for software/hardware level efficiency.
• Designed hardware Accelerator using look-up table approach for NeRF based 3D scene rendering.
• Achieved high average PSNR 31.70 and reduced the rendering time to 102s for a complete scene at the hardware-level.
• Skills learned: Low-precision representation, fixed gird-based quantization, hardware accelerator design, 3D Gaussian representation, lookup tables, and efficient NeRF rendering.
• Tools learned: GPU/Cuda, Modelsim and Synopsys.

LT-SNN: Spiking Neural Network with Learnable Threshold for Event-based Classification and Object Detection [Code]

• Designed a shallow Yolo-SNN architectures for event-based classification object detection.
• Implemented layer-wise learnable threshold in SNN for optimal learning.
• Implemented sigmoid gradient surrogation for backward pass in SNN.
• Implemented low-precision representation of weights and activations using 4-bit quantization of membrane potential.
• Achieved state-of-the-art accuracy 80.7 on DVS-CIFAR10 and mean avergae precision value (mAP) 0.30 on Prophesee Gen1 dataset.
• Skills learned: Event-based processing, SNN architecture design, surrogate gradient methods, low-precision quantization, and performance optimization for spiking neural networks.
• Tools learned: Event/DVS sensros and GPU training.

Quantization methods for different architectures including CNN, SNN and NeRF [Code]

• Designed weight and activation quantization approaches to support low-prcision representation of different architectures.
• Implemented SAWB, Adaptive Rounding and fixed-gird for weight quantization
• Implemented PACT, fixed grid based and STE for activations quantization.
• This work supports different architectures including CNNs, SNNs and 3D scene rendering architectures.
• Skills learned: Low-precision training and inference, quantization, and resource optimization.
• Tools learned: Quantization and GPU training.

Low-precision-custom-SNN-Yolov2 for Object detection using Prophesee Gen1 and Gen4 Datasets [Code]

• Designed a shallow Yolo-SNN architectures for event-based object detection.
• Prepared Prophesee Gen1 and Gen4 data for object detection.
• Implemented low-precision representation of weights and activations using 4-bit and 2-bit quantization of membrane potential.
• Achieved state-of-the-art mean avergae precision value (mAP) 0.30 for Prophesee Gen1 dataset.
• Validated the shallow-Yolo-SNN on Prophesee Gen4 (most complex event data to date) and achieved 0.24 mAP.
• Skills learned: Low-precision training and inference, quantization, and object detection and tracing.
• Tools learned: Quantization, IOU metrices and GPU training.

SpQuant-SNN: ultra-low precision membrane potential with sparse activations unlock the potential of on-device spiking neural networks applications[Code]

• Designed a 3-bit precision integer-only quantization scheme for membrane potential for VGG-SNN and Yolo-SNN architectures.
• Designed the complete SNN quantization flow to compress the models.
• Implemented Spatial-Channel pruning with fully quantized SNN to develop SpQuant-SNN for efficient edge computations.
• Benchmarked SpQuant-SNN on complex datasets with >3X memory and >5X FLOPs reduction within less than 1% accuracy drop.
• This work is accepted in Frontier of Neuroscience, 2024. The code will be available soon.
• Skills learned: Low-precision training and inference, quantization, and object classification, detection and tracing.
• Tools learned: Quantization, accuracy and IOU metrices and GPU training and inference.

Self-supervised Learning with Vision Transformers for Downstream Tasks

• Implemented mainstream vision transformer architectures for better feature learning on unlabeled data
• Simplified the pre-trained model to develop its tiny version for hardware-level translation.
• Pre-trained the model on ImageNet-100 and ImageNet-1000 dataset and validated the slight imprivement in comparison to the currest SOTA works
• This project is accepted in MLSys, 2024.
• Skills learned: Vision transformers, self-supervised learning, model compression, hardware-level optimization, and dataset evaluation.

Low Precision CNN-based Architecture Design for Information Processing from Event-based Camera [Code]

• Designed low-precision sparse autoencoder architecture for Event (Prophesee-Gen1 and Prophesee-Gen4) data compression.
• Designed log of two-based quantization module to covert 32-bit weights and activations to 4-bit and 2-bit precision.
• Achieved high accuracy >91% and mean average precision 0.30 with >10x image data compression for both Prophesee-Gen1 and Prophesee-Gen4 dataset.
• Designed IMC-based hardware using proposed sparse and fully quantized autoencoder architecture for air traffic control applications.
• This project is accepted in Asilomar, 2022.
• Skills learned: Sparse autoencoder design, VAEs design, low-precision quantization, event-based data compression, IMC hardware design, and air traffic control applications.

UNet based Image Segmentation Architecture Design and Convex hull implementation on segmented image for area extraction [Code]

• Designed shallow UNet Architecture with low-precision weights and activations for image segmentation.
• Extracted convex hull to compute the object polygon area.
• Achieved 0.99 IOU for the custom object dataset.
• Skills learned: Low-precision deep learning, UNet architecture design, image segmentation, convex hull computation, and accuracy evaluation.
• Tools learned: Quantization and GPU training and inference.

Optimization of Prefill Time/ Time to First Token Generation of Mamba (SSM) for Faster On-device Inference

• I have worked on this project during my internship at Samsung Research America, AI Center
• Extended work on this project is still in progress

Estimation of Local Layout Effect (LLEs) using Machine Readable Specs (MRS) for Design Technology Team, Intel Corporations

• Using LLE rules from the QA team, designed machine-readable specs to estimate the LLEs’ presence in the layouts.
• Translated MRS into Python utility and validated automated LLE estimation using different layouts.
• Provided prototype version of python-based utility to QA and TTR team for small and large-scale layout testing.
• Documented the whole project and handed it over to the DE and QA team.
• Algorithm implementation is not availble due to industry rights.

Python-Based Automated Layout Generation for Different Cell Types, Intel Corporations

• Developed Python algorithm to generate automatic and DRC clean layouts.
• Generated and delivered layouts using a designed algorithm for LLE test cases, standard cells, and memory cells.
• Documented the whole project and handed it over to the DE and QA team.
• Algorithm implementation is not availble due to industry rights.

Pre-production Automated Verification and Completion of Design Run-sets, Intel Corporations

• Implemented an algorithm to identify the key requirements and discrepancies of run-sets for auto-correction.
• Documented the whole project and handed it over to the DE team.
• Algorithm implementation is not availble due to industry rights.

Techon 2022

• Presented our group research on learnable potential threshold based Spiking neural Netwoeks.

CBRIC Annual Review 2022

• Presented our group research on scalability of Spiking Neural Networks for computer vision applications.

Asilomar 2022

• Presented my research paper "Spatial-temporal Data Compression of Dynamic Vision Sensor Output with High Pixel-level Saliency using Low-precision Sparse Autoencoder".

COCOSYS Annual Review 2023

• Presented our group research on compression and pruning of Spiking Neural Networks for computer vision applications.

NeurIPS 2023

• Attended coference to represent my research group and participated in different workshops related to stable diffusion.

IJCNN 2024

• Presented my research work on Spiking Neural Networks with learnable threshold for computer vision applications.

ICONS 2024

• Presented our group research on compression and pruning of Spiking Neural Networks for computer vision applications.