Other Models
Collection
91 items
β’
Updated
RAHUL YASHWANTKUMAR GUPTA
ryg81
AI & ML interests
None yet
Recent Activity
updated
a collection
about 4 hours ago
Other Models
updated
a collection
about 4 hours ago
Other Models
updated
a collection
about 8 hours ago
Other Models
Organizations
None yet
ryg81's activity
updated
a
collection
about 4 hours ago
updated
a
collection
about 8 hours ago
updated
a
collection
about 20 hours ago
reacted to
Duskfallcrew's
post with ππ₯
3 days ago
Post
2995
Just been starting to port my articles over that mattered most to me from Civitai.
Look, i'm not going to sit here and whine, complain and moan entirely - they know why i've left, they're going to thrive without me.
I'm a mere spec compared to their future, and that's amazing.
But the journey continues, i've posted my Design 101 for Ai - the first one up -- i BELEIVE it's the first one, as it delves back to how Arts and Crafts connect to AI.
I'm still looking for a model hub in future for my insane 800+ models i'd published - considering that that's half of what i've got sitting in my repos on HF.
Look, i'm not going to sit here and whine, complain and moan entirely - they know why i've left, they're going to thrive without me.
I'm a mere spec compared to their future, and that's amazing.
But the journey continues, i've posted my Design 101 for Ai - the first one up -- i BELEIVE it's the first one, as it delves back to how Arts and Crafts connect to AI.
I'm still looking for a model hub in future for my insane 800+ models i'd published - considering that that's half of what i've got sitting in my repos on HF.
reacted to
retronic's
post with π₯
4 days ago
Post
4257
Colox, a reasoning AI model. I am currently working on a model smarter than GPT o1 that thinks before it speaks. It is coming tomorrow in the afternoon.
- 7 replies
updated
a
collection
4 days ago
reacted to
ahmed-masry's
post with π
6 days ago
Post
5060
Happy to announce AlignVLM π β a novel approach to bridging vision and language latent spaces for multimodal understanding in Vision-Language Models (VLMs) πππΌ
π Read the paper: AlignVLM: Bridging Vision and Language Latent Spaces for Multimodal Understanding (2502.01341)
π§ Whatβs the challenge?
Aligning visual features with language embeddings remains a major bottleneck in VLMs. Existing connectors such as Multi-layer perceptron (MLPs) often introduce noise that degrades performance. β
π― Our Solution: ALIGN Connector
We propose AlignVLM, a method that maps vision features into a weighted average of LLM text embeddings, ensuring they remain in a space that the LLM can effectively interpret. β
π¬ How does it perform?
We compared ALIGN against common connectors like MLPs, Perceiver Resampler, and Ovis trained under similar configurations. The results? ALIGN outperforms them all π on diverse document understanding tasks π.
π Meet the AlignVLM Model Family!
We trained Llama 3.1 (1B, 3B, 8B) using our connector and benchmarked them against various models. The results:
β AlignVLM surpasses all Base VLMs trained under similar configurations. β Our models also perform competitively against Instruct VLMs such as Qwen2-VL and InternVL-2.5 π.
π€ What about robustness to noise?
We injected Gaussian noise (ΞΌ=0, Ο=3) into the vision encoderβs outputs before feeding them to the connector:
β ALIGN Connector: Minimal drop (β1.67%) β proving its high robustness!
β MLP Connector: Severe degradation (β25.54%) β struggling with noisy inputs.
Code & model weights coming soon! Stay tuned! π₯
π Read the paper: AlignVLM: Bridging Vision and Language Latent Spaces for Multimodal Understanding (2502.01341)
π§ Whatβs the challenge?
Aligning visual features with language embeddings remains a major bottleneck in VLMs. Existing connectors such as Multi-layer perceptron (MLPs) often introduce noise that degrades performance. β
π― Our Solution: ALIGN Connector
We propose AlignVLM, a method that maps vision features into a weighted average of LLM text embeddings, ensuring they remain in a space that the LLM can effectively interpret. β
π¬ How does it perform?
We compared ALIGN against common connectors like MLPs, Perceiver Resampler, and Ovis trained under similar configurations. The results? ALIGN outperforms them all π on diverse document understanding tasks π.
π Meet the AlignVLM Model Family!
We trained Llama 3.1 (1B, 3B, 8B) using our connector and benchmarked them against various models. The results:
β AlignVLM surpasses all Base VLMs trained under similar configurations. β Our models also perform competitively against Instruct VLMs such as Qwen2-VL and InternVL-2.5 π.
π€ What about robustness to noise?
We injected Gaussian noise (ΞΌ=0, Ο=3) into the vision encoderβs outputs before feeding them to the connector:
β ALIGN Connector: Minimal drop (β1.67%) β proving its high robustness!
β MLP Connector: Severe degradation (β25.54%) β struggling with noisy inputs.
Code & model weights coming soon! Stay tuned! π₯
looks good, but will they put for us to use or just showing their development and teasing us. :)
updated
a
collection
7 days ago
updated
a
collection
8 days ago
getting error
8
#1 opened 12 days ago
by
ryg81
updated
2
collections
10 days ago
reacted to
fuzzy-mittenz's
post with π₯
11 days ago
Post
2597
Not many seemed to notice but what was probably meant to be a WIN for artist's rights in the US Office of Copyright has solved some fundamental issues for the community.
In our recent article I outline how Companies like Suno, OpenAI, Midjourney etc can no longer claim any right to copy your work that you create with their platforms
We also look at other ways this study and new rules for AI will fundamentally effect creators who use it and companies incentives to give them control over certain aspects might change because of this. it's broken down pretty well here: https://huggingface.co/blog/fuzzy-mittenz/copyright-in-ai
In our recent article I outline how Companies like Suno, OpenAI, Midjourney etc can no longer claim any right to copy your work that you create with their platforms
We also look at other ways this study and new rules for AI will fundamentally effect creators who use it and companies incentives to give them control over certain aspects might change because of this. it's broken down pretty well here: https://huggingface.co/blog/fuzzy-mittenz/copyright-in-ai
commented on
KV Caching Explained: Optimizing Transformer Inference Efficiency
11 days ago
Can this be similar for image generation models? (I am not a programmer :- or expert in AI))
updated
a
collection
14 days ago