LeroyDyer/_Spydaz_Web_AGI_DeepThink_R1

Quote for Motivation:

"Success comes from defining each task in achievable steps. Every completed step is a success that brings you closer to your goal. If your steps are unreachable, failure is inevitable. Winners create more winners, while losers do the opposite. Success is a game of winners!"

"To grow as a professional, set goals just beyond your current abilities. Achieving these milestones will not only overcome obstacles but also strengthen your skillset. If your tasks are too easy, you’ll never challenge yourself or improve, and life will pass you by!"

— # Leroy Dyer (1972-Present)

DEEP THINK

deepseek model !

truly found the formula! this model does get carried away with its thinking : but it comes to the right answer ! it has super long reasoning traces and uses test driven devlopment methods !

SpydazWeb Mistral Model :

This model has been trained to perform with contexts of 512k , although in training it has been trained mainly with the 2048 for general usage : the long context aspect also allows fro advanced projects and sumarys as well as image and audio translationns and generations:

Highly trained as well as methodolgy oriented , this model has been trained on the reAct Prcess and other structured processes . hence structured outputs (json) are very highly trained as well as orchestration of other agents and tasks : the model has been trained for tools use as well as funtion use : as well as custom processes and tools : some tools do not need code either as thier implication means the model may even generate a tool or artifact to perfrom the task :

Training :

For this model :
the focus has been mainly on methodology :

• Chain of thoughts
• step by step planning
• tree of thoughts
• forest of thoughts
• graph of thoughts

Self-Correction and Thought Processes:

Reward Modelling : the model was reward modelled using the varioius evaluation datasets : as this seemed to be a good method for determining and embedding the natural llangueg and ground truth datasets : Its important to use ground truth datasets when doing grpo Training : A variety of structured outputs were rewarded , so the model was rewarded for planning , reasoning , explaining , thining , sharing its thoughts : as well as answering . here the focus is on the route to the finished tas as distilling inforamtion from other models only mimics thier outcomes with no understanding : reward training solves this :

Producing explanations and planning etc this enables the model to begin to thin about the different possible routes to the ansers given : So it is important not to highly fit data but to obtain a steady average across the whole datsets : if the model moves drastically in any directing or begins to converge to lower loss level we stop ! ( very important as we want generalisation across multiple tas and datsets )

hidden techniques of training this model :

Focused Tasks:

Training was task-based, with a limited number of highly specific samples (e.g., 4k samples per task) to prioritize depth over breadth. Tasks included interpreting spectrograms, ECG images, SMILES chemical compounds, charts, and diagrams rather than general-purpose images.

Overfitting for Baseline Embeddings:

Initial heavy overfitting on large parameter stacks ensured robust embeddings, forming a strong base for subsequent fine-tuning. Training Techniques:

Deep Training:

Adjusted the entire model to create a strong foundation.

Shallow Training:

Focused on specific layers to refine task-specific capabilities. Attention-Head Training: Allowed specific attention heads to specialize in task-relevant features while preserving other model capacities.

Prompting :

A Basic Prompting method :useful for mistral models :

def GetPrompt_(Input: str,Instruct: str = ""):
    def FormatMistralPrompt(Instruct: str,Input: str):
        Prompt: str = f"""<s><INST>{Instruct}</INST>{Input}</s>"""
        return Prompt
    def CreatePrompt_Standard(Prompt:str, SystemPrompt: str = "You are the World Archive a Helpfull AI System , Answering questions and performing tasks: " ):
        IPrompt : str = f"""{SystemPrompt}
            ### Instruction : Answer all questions Expertly and professionally : 
                            you are expertly qualified to give any advice or provide any solutions:
                            your experience as a life coach and mentor as well as system designer and python developer, 
                            will enable you to answer these questions :Think logically first, think object oriented , 
                            think methodology bottom up or top down solution. before you answer. 
                            think about hthe user intent for this this problem and select the correct methodology.
                            Using the methodogy solve each stage , step by step, error check your work.
                            Before answering adusting your solution where required.
                             consider any available tools: return the response formatted in markdown:



            ### Input
            {Prompt}
            ### Response : """

        return IPrompt

    MistralPrompt = FormatMistralPrompt(Instruct,Input)
    prompt = CreatePrompt_Standard(MistralPrompt,)
    return prompt

BASE64 : Encode / Decode for multi modal files

import base64
from pathlib import Path

def encode_file_to_base64(input_file_path: str, output_file_path: str = None) -> str:
    """
    Encodes any file (image or sound) to Base64.
    
    Args:
        input_file_path (str): Path to the input file.
        output_file_path (str): Optional path to save the Base64 encoded string.
        
    Returns:
        str: Base64 encoded string of the file.
    """
    file_path = Path(input_file_path)
    if not file_path.is_file():
        raise FileNotFoundError(f"File not found: {input_file_path}")
    
    # Read file in binary mode
    with open(file_path, "rb") as file:
        file_data = file.read()
    
    # Encode to Base64
    base64_data = base64.b64encode(file_data).decode('utf-8')
    
    # Save to output file if specified
    if output_file_path:
        with open(output_file_path, "w") as output_file:
            output_file.write(base64_data)
    
    return base64_data

def decode_base64_to_file(base64_data: str, output_file_path: str):
    """
    Decodes a Base64 string back into its original binary file.
    
    Args:
        base64_data (str): The Base64 encoded string.
        output_file_path (str): Path to save the decoded file.
    """
    # Decode Base64 to binary data
    file_data = base64.b64decode(base64_data)
    
    # Write binary data to the output file
    with open(output_file_path, "wb") as file:
        file.write(file_data)

# Encode sound file to Base64
encoded_sound = encode_file_to_base64("example.mp3", "example_base64.txt")
print(f"Encoded sound file saved to example_base64.txt")

# Decode Base64 back to sound file
decode_base64_to_file(encoded_sound, "decoded_example.mp3")
print("Decoded sound file saved as decoded_example.mp3")

# Encode image file to Base64
encoded_image = encode_file_to_base64("example_image.jpg", "example_image_base64.txt")
print(f"Encoded image file saved to example_image_base64.txt")

# Decode Base64 back to image file
decode_base64_to_file(encoded_image, "decoded_example_image.jpg")
print("Decoded image file saved as decoded_example_image.jpg")

Explanation of the Functions

Encoding Pipeline: Read the file as binary (rb mode). Use base64.b64encode() to encode the binary data into Base64 format. Save the encoded string to an optional file if required.

Decoding Pipeline: Decode the Base64 string back to binary using base64.b64decode(). Save the binary data as the output file in its original format.

Notes These functions can handle any binary file, including sound files (MP3, WAV, OGG) and image files (JPG, PNG, BMP). The Base64 output can be used in text-based applications or embedded in HTML/JSON as needed. Ensure the input file exists, and specify the correct output path during decoding. This design is flexible and reusable for various file types, making it a robust solution for encoding and decoding files into Base64.

Converting DataSets:

# Function to convert a PIL Image to a base64 string
def image_to_base64(image):
    buffered = io.BytesIO()
    image.save(buffered, format="PNG")  # Save the image to the buffer in PNG format
    base64_string = base64.b64encode(buffered.getvalue()).decode('utf-8')
    return base64_string


# Define a function to process each example in the dataset
def process_images_func(examples):

    texts = examples["text"]
    images = examples["image"]  # Assuming the images are in PIL format

    # Convert each image to base64
    base64_images = [image_to_base64(image) for image in images]

    # Return the updated examples with base64-encoded images
    return {
        "text": texts,
        "image_base64": base64_images  # Adding the Base64 encoded image strings
    }

# Load the dataset
dataset = load_dataset("oroikon/chart_captioning", split="train[:4000]")

# Process the dataset by converting images to base64
processed_dataset = dataset.map(process_images_func, batched=True)