{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pe53IRkGICfI"
      },
      "source": [
        "# Laboratorio Redes Neuronales (parte 2)\n",
        "\n",
        "En este laboratorio vamos a ver algunos ejemplos más de uso de Keras para resolver problemas con redes neuronales.\n",
        "\n",
        "Primero vamos a ver un módulo, Keras Tuner ([link](https://keras.io/guides/keras_tuner/getting_started/)) que permite buscar, mediante diferentes métodos los mejores hiperparámetros, una tarea bastante tediosa.\n",
        "\n",
        "Vemos que la búsqueda de hiperparámetros puede ser un poco tediosa. Para ello keras provee Keras Tuner , que simplifica la búsqueda manual, mediante diferentes métodos de búsqueda, que también pueden elegirse."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "hPHXLeBnICfJ"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "from tensorflow import keras\n",
        "from keras.datasets import mnist\n",
        "import matplotlib.pyplot as plt\n",
        "from sklearn.metrics import classification_report\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "h5e7sPwdICfK",
        "outputId": "4d0fabfc-c6c5-433e-e383-3f17cf43a573"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "((60000, 28, 28), (10000, 28, 28))"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ],
      "source": [
        "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
        "x_train.shape, x_test.shape"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Normalizamos..."
      ],
      "metadata": {
        "id": "SevSMsnIJUpa"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "a3m1yNMqICfP"
      },
      "outputs": [],
      "source": [
        "# Utilizando la operacion reshape, convertir las imagenes de matriz a vector\n",
        "x_train1 = x_train.reshape(60000,28*28)\n",
        "x_test1 = x_test.reshape(10000,28*28)\n",
        "# Convertir el tipo de los datos de tipo np.uint8 a np.float32\n",
        "x_train1 = x_train1.astype(np.float32)\n",
        "x_test1 = x_test1.astype(np.float32)\n",
        "# Dividimos el valor de cada pixel entre 255. para asegurarnos que todos los pixeles queden en el rango $[0,1]$\n",
        "x_train1 = x_train1 / 255.0\n",
        "x_test1 = x_test1 / 255.0\n",
        "# Restamos a cada pixel 0.5 para asegurarnos que todos los pixeles queden en el rango $[-0.5,0.5]$\n",
        "x_train1 = x_train1 - 0.5\n",
        "x_test1 = x_test1 - 0.5\n",
        "\n",
        "####################################################################################################################\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "I5mkY_1C3w0-"
      },
      "outputs": [],
      "source": [
        "# Cuando verificamos que quedó bien, modificamos los datos originales\n",
        "x_train = x_train1\n",
        "x_test = x_test1\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bdyEvKsoICfQ"
      },
      "source": [
        "Separamos entrenamiento y validación"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "3q4dl7gnICfQ"
      },
      "outputs": [],
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "x_train, x_val, y_train, y_val = train_test_split(x_train, y_train, test_size=0.1, random_state=0)\n",
        "\n",
        "# Verificamos que todo este bien\n",
        "assert len(x_train) == len(y_train)\n",
        "assert len(x_val) == len(y_val)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Primero instalamos Keras Tuner..."
      ],
      "metadata": {
        "id": "EsYo9D1SJ8Rx"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "nxiJEZSD7XsY",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "8f90de08-85ef-440d-abe5-8d11d062f379"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting keras-tuner\n",
            "  Downloading keras_tuner-1.3.5-py3-none-any.whl (176 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m176.1/176.1 kB\u001b[0m \u001b[31m4.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from keras-tuner) (23.1)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from keras-tuner) (2.31.0)\n",
            "Collecting kt-legacy (from keras-tuner)\n",
            "  Downloading kt_legacy-1.0.5-py3-none-any.whl (9.6 kB)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->keras-tuner) (3.2.0)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->keras-tuner) (3.4)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->keras-tuner) (2.0.4)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->keras-tuner) (2023.7.22)\n",
            "Installing collected packages: kt-legacy, keras-tuner\n",
            "Successfully installed keras-tuner-1.3.5 kt-legacy-1.0.5\n"
          ]
        }
      ],
      "source": [
        "!pip install keras-tuner --upgrade\n",
        "import keras_tuner as kt"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Supongamos que queremos probar una red como la que trabajamos en la clase anterior para reconocer dígitos, y nos interesa saber si funciona mejor una relu o una sigmoid.\n",
        "\n",
        "Para utilizar Keras Tuner vamos primero a crear una función que define el modelo, recibiendo los hiperparámetros como argumento (en este caso,  definimos que la función de activación admite dos valores, y que la cantidad de unidades de la capa puede ser entre 1 y 32):\n",
        "\n"
      ],
      "metadata": {
        "id": "eoi7xpxdjYdE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def build_model(hp):\n",
        "  activation1 = hp.Choice('activation1',['relu','sigmoid'] )\n",
        "  units1 = hp.Int(\"units\", min_value=1, max_value=512, step=1)\n",
        "\n",
        "  model = keras.Sequential()\n",
        "  model.add(keras.layers.Dense(input_dim=(x_train.shape[1]),\n",
        "                               units =units1, activation = activation1))\n",
        "  model.add(keras.layers.Dense(10, activation=\"softmax\", name=\"output_layer\"))\n",
        "\n",
        "  model.compile(optimizer=keras.optimizers.SGD(0.001), loss='sparse_categorical_crossentropy',\n",
        "                metrics=['accuracy'])\n",
        "  return model\n"
      ],
      "metadata": {
        "id": "XfiaEZ0cjSbt"
      },
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Vamos a definir luego el tuner, que permite probar diferentes modelos y elegir el mejor. Para eso, le vamos a decir que utilice como objetivo a minimizar la pérdida en el conjunto de validación, y un máximo de 5 intentos, a partir del modelo parametrizado que definimos en el paso anterior. El tuner que vamos a utilizar es una búsqueda aleatoria (Keras Tuner ofrece otras opciones)"
      ],
      "metadata": {
        "id": "cObTz4e5lSXu"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "tuner = kt.RandomSearch(build_model, objective='val_loss', max_trials=5, overwrite=True,\n",
        "    directory=\"my_dir\")\n",
        "tuner.search_space_summary()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_lxFHudRlRWS",
        "outputId": "4e5a097e-46ba-4197-f185-0d1670c50df5"
      },
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Search space summary\n",
            "Default search space size: 2\n",
            "activation1 (Choice)\n",
            "{'default': 'relu', 'conditions': [], 'values': ['relu', 'sigmoid'], 'ordered': False}\n",
            "units (Int)\n",
            "{'default': None, 'conditions': [], 'min_value': 1, 'max_value': 512, 'step': 1, 'sampling': 'linear'}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Probamos con el tuner un aprendizaje con 5 épocas (observar que no llamamos a compile ni a try)"
      ],
      "metadata": {
        "id": "HJnFBtEtmly5"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "tuner.search(x_train, y_train, epochs=5, validation_data=(x_val, y_val))\n",
        "best_model = tuner.get_best_models()[0]\n",
        "\n",
        "tuner.results_summary()\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zXR3gmpamz04",
        "outputId": "53c867c4-96ca-4913-d5df-bc0ff0faab49"
      },
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Trial 5 Complete [00h 00m 42s]\n",
            "val_loss: 1.2482744455337524\n",
            "\n",
            "Best val_loss So Far: 0.45816174149513245\n",
            "Total elapsed time: 00h 03m 16s\n",
            "Results summary\n",
            "Results in my_dir/untitled_project\n",
            "Showing 10 best trials\n",
            "Objective(name=\"val_loss\", direction=\"min\")\n",
            "\n",
            "Trial 2 summary\n",
            "Hyperparameters:\n",
            "activation1: relu\n",
            "units: 376\n",
            "Score: 0.45816174149513245\n",
            "\n",
            "Trial 3 summary\n",
            "Hyperparameters:\n",
            "activation1: relu\n",
            "units: 167\n",
            "Score: 0.46321651339530945\n",
            "\n",
            "Trial 0 summary\n",
            "Hyperparameters:\n",
            "activation1: sigmoid\n",
            "units: 501\n",
            "Score: 1.2457010746002197\n",
            "\n",
            "Trial 4 summary\n",
            "Hyperparameters:\n",
            "activation1: sigmoid\n",
            "units: 477\n",
            "Score: 1.2482744455337524\n",
            "\n",
            "Trial 1 summary\n",
            "Hyperparameters:\n",
            "activation1: sigmoid\n",
            "units: 53\n",
            "Score: 1.542050838470459\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Vemos el resultado de nuestro mejor modelo contra el conjunto de testeo"
      ],
      "metadata": {
        "id": "ju7GufByni_-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "\n",
        "predictionstest = best_model.predict(x_test)\n",
        "y_test_pred = np.argmax(predictionstest, axis=1)\n",
        "print(classification_report(y_test, y_test_pred))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "umecB5fnndwC",
        "outputId": "6f297fff-f687-4a93-f345-a6223a530782"
      },
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "313/313 [==============================] - 0s 1ms/step\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.91      0.97      0.94       980\n",
            "           1       0.92      0.96      0.94      1135\n",
            "           2       0.89      0.84      0.87      1032\n",
            "           3       0.86      0.88      0.87      1010\n",
            "           4       0.87      0.89      0.88       982\n",
            "           5       0.84      0.80      0.82       892\n",
            "           6       0.89      0.91      0.90       958\n",
            "           7       0.90      0.89      0.89      1028\n",
            "           8       0.87      0.81      0.84       974\n",
            "           9       0.85      0.86      0.85      1009\n",
            "\n",
            "    accuracy                           0.88     10000\n",
            "   macro avg       0.88      0.88      0.88     10000\n",
            "weighted avg       0.88      0.88      0.88     10000\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lsbXYC-z7egx"
      },
      "source": [
        "*Se pide*: Implementar una búsqueda de hiperparámetros con RandomSearch, utilizando:\n",
        "\n",
        "*   Red de dos capas ocultas\n",
        "*   Optimizador SGD\n",
        "*   Entrenamiento en la búsqueda de 10 épocas\n",
        "*   Máximo de intentos: 20\n",
        "\n",
        "Variar:\n",
        "\n",
        "*   Learning rate de SGD entre 1e-4 y 1e-2 (valor real)\n",
        "*   funciones de activación de capas ocultas: ['relu', 'sigmoid']\n",
        "*   Cantidad de neuronas de las capas: [8, 32, 64]\n",
        "\n",
        "Mostrar métricas para el mejor modelo obtenido"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4"
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.6"
    },
    "accelerator": "GPU"
  },
  "nbformat": 4,
  "nbformat_minor": 0
}