JARVIS

What is JARVIS

JARVIS is an innovative project aimed at advancing artificial general intelligence (AGI) through collaborative systems that integrate large language models (LLMs) with expert AI models. Its primary purpose is to facilitate complex AI task execution by enabling seamless interaction between users and a variety of AI models hosted on platforms like Hugging Face. Targeted at researchers, developers, and AI enthusiasts, JARVIS addresses the challenge of efficiently managing and executing diverse AI tasks through a structured, multi-stage workflow. The project is built using Python and leverages key technologies such as PyTorch, Hugging Face Transformers, and Gradio for user interface development.

How to Use

Installation Instructions

1. System Requirements: Ensure you are using Ubuntu 16.04 LTS or later, with a minimum of 12GB RAM and 24GB VRAM recommended for optimal performance.
2. Environment Setup:
   • Create a new conda environment:

     conda create -n jarvis python=3.8
     conda activate jarvis
     

   • Install required dependencies:

     conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia
     pip install -r requirements.txt
     

3. Configuration: Replace openai.key and huggingface.token in server/configs/config.default.yaml with your OpenAI API key and Hugging Face token, or set them as environment variables.
4. Running the Server:
   • Download models using:

     cd models
     bash download.sh
     

   • Start the server:

     python models_server.py --config configs/config.default.yaml
     python awesome_chat.py --config configs/config.default.yaml --mode server
     

5. Accessing the API: Use the provided endpoints (/hugginggpt, /tasks, /results) to interact with JARVIS.

Usage Examples

• API Request:

  curl --location 'http://localhost:8004/tasks' \
  --header 'Content-Type: application/json' \
  --data '{"messages":[{"role":"user","content":"Generate an image based on /examples/d.jpg."}]}'
  

Documentation Links

For more detailed instructions, refer to the official documentation.

Key Features

• Multi-Stage Workflow: JARVIS operates through a four-stage process: Task Planning, Model Selection, Task Execution, and Response Generation, ensuring efficient task management.
• Integration with Hugging Face: Seamlessly connects with various expert models from the Hugging Face Hub, allowing for diverse AI task execution.
• Flexible Inference Modes: Supports local, Hugging Face, and hybrid inference modes, catering to different deployment needs and resource availability.
• User-Friendly Interfaces: Offers a web interface and CLI mode for easy interaction, making it accessible for users with varying technical expertise.
• Docker Support for Jetson Devices: Provides a Dockerfile for NVIDIA Jetson devices, enabling deployment on edge devices with GPU acceleration.
• Community and Enterprise Features: Includes extensibility options for developers to integrate additional models and functionalities, fostering community contributions.
• Performance Optimization: Designed to handle large-scale tasks with efficient resource management, ensuring scalability for enterprise applications.
• Real-Time Collaboration: Facilitates real-time interactions between users and AI models, enhancing the user experience and task execution speed.

JARVIS stands out as a comprehensive solution for leveraging AI in complex task automation, making it a valuable tool for developers and researchers in the AI domain.