{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lab42"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From f:\\KTU\\Neuroninių tinklų metodai\\venv\\Lib\\site-packages\\keras\\src\\losses.py:2976: The name tf.losses.sparse_softmax_cross_entropy is deprecated. Please use tf.compat.v1.losses.sparse_softmax_cross_entropy instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "f:\\KTU\\Neuroninių tinklų metodai\\venv\\Lib\\site-packages\\tqdm\\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from time import time\n",
    "from sklearn.model_selection import train_test_split as split_test_train\n",
    "\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, SimpleRNN, LSTM, GRU\n",
    "from keras.optimizers import RMSprop\n",
    "from keras.callbacks import Callback\n",
    "import tensorflow_datasets as tfds\n",
    "import tensorflow as tf\n",
    "\n",
    "tfds.disable_progress_bar()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "text:  b\"This was an absolutely terrible movie. Don't be lured in by Christopher Walken or Michael Ironside. Both are great actors, but this must simply be their worst role in history. Even their great acting could not redeem this movie's ridiculous storyline. This movie is an early nineties US propaganda piece. The most pathetic scenes were those when the Columbian rebels were making their cases for revolutions. Maria Conchita Alonso appeared phony, and her pseudo-love affair with Walken was nothing but a pathetic emotional plug in a movie that was devoid of any real meaning. I am disappointed that there are movies like this, ruining actor's like Christopher Walken's good name. I could barely sit through it.\"\n",
      "label:  0\n",
      "texts:  [b'Closer to reality and containing more depth than \"Breakdance\", Stan Lathan\\'s \"Beat Street\" is still a pretty dull show. Again this pic is really only cashing in on the \\'breakin\\' craze but at least we get a little bit of entertainment from the plot, which concerns the lives of three young friends and a younger brother, all growing up in Queens, New York. Each has their own unique talent. One is a hustler, one a d.j., another an artist who creates \\'burners\\', while little bro\\' Lee is a hot \\'hip hop\\' dancer.<br /><br />Lathan is unable to generate any real audience interest in the story though, and his young cast are likewise struggling with their characters. Therefore it is left almost entirely to the funky music and the \\'fresh\\' dancing to save the day.<br /><br />Choreography is again sharp for both club and street scenes, but this alone is not enough to lift \"\"Beat Street\" to greater heights. Unfortunately the film really falls flat late on, after showing a glimmer of hope that it just might get interesting.<br /><br />Sunday, August 25, 1996 - Video'\n",
      " b'Of the three remakes of this plot, I like them all, I have all three on VHS and in addition have a copy of this one on DVD. There is just enough variation in the scripts to make all three entertaining and re-watchable. In addition has any other film been remade three times with such all star casts in each? Of course the main stars in this one are great, but the supporting actors are also superb. I particularly like William Tracy as Pepi. He was such a scene stealer that I have searched to find other movies he is in. He appeared in many, but most are not available. As the other comments, I also say - buy this one.'\n",
      " b\"This Cecil B. DeMille epic of the old West contains what may be Jean Arthur's finest performance, as a hysterical, eccentric, incurably amoral, but devotedly doting Calamity Jane. She really pulled it off! Gary Cooper is at his most taciturn, but manages some occasional pithy sayings: 'The plains are big, but trails cross ... sometimes.' The story is a pastiche to end all pastiches. All the cowboy heroes of Western lore seem to be in there somehow except for Jesse James. Even Abraham Lincoln opens the story in person (or at least, DeMille would have us believe so). There is no room for anything so evanescent as subtlety, this is a 'stomp 'em in the face' tale for the masses. A remarkable thing about this film however is that it is a very early full frontal attack on what Eisenhower was eventually to name 'the military industrial complex'. It isn't just a story about gun-runners, but about arming anyone for money, and doing so from the heart of Washington. But let's not get into politics, let's leave that to DeMille, who can be guaranteed to be superficial. The chief interest of this film all these years later is that it uses the first film score composed by George Antheil, who has a lot to say about the job in his autobiography, 'Bad Boy of Music'. Antheil seems to have originated 'the big sound' adopted by all subsequent Westerns, whereby the plains sing out with the voices and sounds of countless cowboys in the sky, celebrating the open spaces and interweaving common melodies. That is why it does not sound at all unusual, because we have heard it a thousand times. But he seems to have been the first to summon up the combined rustlings of all the sage brush into this symphony of the open skies which has entered into American mythic lore, and given it a soundtrack which has never varied since then, corny as it may be, but doubtless appropriate. It is amusing to see Anthony Quinn in an early appearance as a Cheyenne Indian. Gabby Hayes is in there somewhere, but you miss him in the crowd. Gary Cooper overtops them all, looming large, - but when did he ever loom small?\"]\n",
      "\n",
      "labels:  [0 1 1]\n"
     ]
    }
   ],
   "source": [
    "def plot_graphs(history, metric):\n",
    "    plt.plot(history.history[metric])\n",
    "    plt.plot(history.history['val_'+metric], '')\n",
    "    plt.xlabel(\"Epochs\")\n",
    "    plt.ylabel(metric)\n",
    "    plt.legend([metric, 'val_'+metric])\n",
    "\n",
    "dataset, info = tfds.load('imdb_reviews', with_info=True, as_supervised=True)\n",
    "train_dataset, test_dataset = dataset['train'], dataset['test']\n",
    "\n",
    "for example, label in train_dataset.take(1):\n",
    "  print('text: ', example.numpy())\n",
    "  print('label: ', label.numpy())\n",
    "\n",
    "BUFFER_SIZE = 10000\n",
    "BATCH_SIZE = 64\n",
    "\n",
    "train_dataset = train_dataset.shuffle(BUFFER_SIZE).batch(BATCH_SIZE).prefetch(tf.data.AUTOTUNE)\n",
    "test_dataset = test_dataset.batch(BATCH_SIZE).prefetch(tf.data.AUTOTUNE)\n",
    "\n",
    "for example, label in train_dataset.take(1):\n",
    "  print('texts: ', example.numpy()[:3])\n",
    "  print()\n",
    "  print('labels: ', label.numpy()[:3])\n",
    "\n",
    "VOCAB_SIZE = 1000\n",
    "encoder = tf.keras.layers.TextVectorization(max_tokens=VOCAB_SIZE)\n",
    "encoder.adapt(train_dataset.map(lambda text, label: text))\n",
    "\n",
    "model = tf.keras.Sequential([\n",
    "    encoder,\n",
    "    tf.keras.layers.Embedding(\n",
    "        input_dim=len(encoder.get_vocabulary()),\n",
    "        output_dim=64,\n",
    "        # Use masking to handle the variable sequence lengths\n",
    "        mask_zero=True\n",
    "    ),\n",
    "    tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(64)),\n",
    "    tf.keras.layers.Dense(64, activation='relu'),\n",
    "    tf.keras.layers.Dense(1)\n",
    "])\n",
    "\n",
    "model.compile(\n",
    "    loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),\n",
    "    optimizer=tf.keras.optimizers.Adam(1e-4),\n",
    "    metrics=['accuracy']\n",
    ")\n",
    "\n",
    "checkpoint = tf.train.Checkpoint(\n",
    "    model=model\n",
    ")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "WARNING:tensorflow:From f:\\KTU\\Neuroninių tinklų metodai\\venv\\Lib\\site-packages\\keras\\src\\engine\\base_layer_utils.py:384: The name tf.executing_eagerly_outside_functions is deprecated. Please use tf.compat.v1.executing_eagerly_outside_functions instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From f:\\KTU\\Neuroninių tinklų metodai\\venv\\Lib\\site-packages\\keras\\src\\engine\\base_layer_utils.py:384: The name tf.executing_eagerly_outside_functions is deprecated. Please use tf.compat.v1.executing_eagerly_outside_functions instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "391/391 [==============================] - 439s 1s/step - loss: 0.6559 - accuracy: 0.5520 - val_loss: 0.5300 - val_accuracy: 0.7615\n",
      "Epoch 2/10\n",
      "391/391 [==============================] - 428s 1s/step - loss: 0.4360 - accuracy: 0.7975 - val_loss: 0.3924 - val_accuracy: 0.8203\n",
      "Epoch 3/10\n",
      "391/391 [==============================] - 852s 2s/step - loss: 0.3608 - accuracy: 0.8432 - val_loss: 0.3666 - val_accuracy: 0.8250\n",
      "Epoch 4/10\n",
      "391/391 [==============================] - 1106s 3s/step - loss: 0.3334 - accuracy: 0.8585 - val_loss: 0.3467 - val_accuracy: 0.8349\n",
      "Epoch 5/10\n",
      "391/391 [==============================] - 716s 2s/step - loss: 0.3153 - accuracy: 0.8655 - val_loss: 0.3263 - val_accuracy: 0.8568\n",
      "Epoch 6/10\n",
      "391/391 [==============================] - 1155s 3s/step - loss: 0.3116 - accuracy: 0.8684 - val_loss: 0.3270 - val_accuracy: 0.8542\n",
      "Epoch 7/10\n",
      "391/391 [==============================] - 1022s 3s/step - loss: 0.3037 - accuracy: 0.8723 - val_loss: 0.3213 - val_accuracy: 0.8578\n",
      "Epoch 8/10\n",
      "391/391 [==============================] - 433s 1s/step - loss: 0.3022 - accuracy: 0.8722 - val_loss: 0.3266 - val_accuracy: 0.8609\n",
      "Epoch 9/10\n",
      "391/391 [==============================] - 430s 1s/step - loss: 0.2968 - accuracy: 0.8750 - val_loss: 0.3257 - val_accuracy: 0.8620\n",
      "Epoch 10/10\n",
      "391/391 [==============================] - 438s 1s/step - loss: 0.2962 - accuracy: 0.8764 - val_loss: 0.3417 - val_accuracy: 0.8557\n"
     ]
    }
   ],
   "source": [
    "history = model.fit(\n",
    "    train_dataset,\n",
    "    epochs=10,\n",
    "    validation_data=test_dataset,\n",
    "    validation_steps=30\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.checkpoint.checkpoint.CheckpointLoadStatus at 0x1a70719f550>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "checkpoint.restore(tf.train.latest_checkpoint(\".\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From f:\\KTU\\Neuroninių tinklų metodai\\venv\\Lib\\site-packages\\keras\\src\\engine\\base_layer_utils.py:384: The name tf.executing_eagerly_outside_functions is deprecated. Please use tf.compat.v1.executing_eagerly_outside_functions instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From f:\\KTU\\Neuroninių tinklų metodai\\venv\\Lib\\site-packages\\keras\\src\\engine\\base_layer_utils.py:384: The name tf.executing_eagerly_outside_functions is deprecated. Please use tf.compat.v1.executing_eagerly_outside_functions instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "391/391 [==============================] - 48s 115ms/step - loss: 0.3387 - accuracy: 0.8605\n",
      "Test Loss: 0.3387344777584076\n",
      "Test Accuracy: 0.8605200052261353\n"
     ]
    }
   ],
   "source": [
    "test_loss, test_acc = model.evaluate(test_dataset)\n",
    "\n",
    "print('Test Loss:', test_loss)\n",
    "print('Test Accuracy:', test_acc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "ename": "FailedPreconditionError",
     "evalue": "./reference is not a directory",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mFailedPreconditionError\u001b[0m                   Traceback (most recent call last)",
      "Cell \u001b[1;32mIn[7], line 1\u001b[0m\n\u001b[1;32m----> 1\u001b[0m \u001b[43mcheckpoint\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43msave\u001b[49m\u001b[43m(\u001b[49m\u001b[43mfile_prefix\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43m \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43m./reference/RNN_chpt\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n",
      "File \u001b[1;32mf:\\KTU\\Neuroninių tinklų metodai\\venv\\Lib\\site-packages\\tensorflow\\python\\checkpoint\\checkpoint.py:2508\u001b[0m, in \u001b[0;36mCheckpoint.save\u001b[1;34m(self, file_prefix, options)\u001b[0m\n\u001b[0;32m   2503\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m   2504\u001b[0m   options\u001b[38;5;241m.\u001b[39mexperimental_write_callbacks\u001b[38;5;241m.\u001b[39mappend(\n\u001b[0;32m   2505\u001b[0m       _update_checkpoint_state_internal\n\u001b[0;32m   2506\u001b[0m   )\n\u001b[1;32m-> 2508\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_write\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m   2509\u001b[0m \u001b[43m    \u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;132;43;01m%s\u001b[39;49;00m\u001b[38;5;124;43m-\u001b[39;49m\u001b[38;5;132;43;01m%d\u001b[39;49;00m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m \u001b[49m\u001b[38;5;241;43m%\u001b[39;49m\u001b[43m \u001b[49m\u001b[43m(\u001b[49m\u001b[43mfile_prefix\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcheckpoint_number\u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m   2510\u001b[0m \u001b[43m    \u001b[49m\u001b[43moptions\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43moptions\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[1;32mf:\\KTU\\Neuroninių tinklų metodai\\venv\\Lib\\site-packages\\tensorflow\\python\\checkpoint\\checkpoint.py:2344\u001b[0m, in \u001b[0;36mCheckpoint._write\u001b[1;34m(self, file_prefix, options)\u001b[0m\n\u001b[0;32m   2342\u001b[0m start_time \u001b[38;5;241m=\u001b[39m time\u001b[38;5;241m.\u001b[39mtime()\n\u001b[0;32m   2343\u001b[0m options \u001b[38;5;241m=\u001b[39m options \u001b[38;5;129;01mor\u001b[39;00m checkpoint_options\u001b[38;5;241m.\u001b[39mCheckpointOptions()\n\u001b[1;32m-> 2344\u001b[0m output \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_saver\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43msave\u001b[49m\u001b[43m(\u001b[49m\u001b[43mfile_prefix\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mfile_prefix\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43moptions\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43moptions\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m   2345\u001b[0m output \u001b[38;5;241m=\u001b[39m _convert_file_name_tensor_to_string(output)\n\u001b[0;32m   2347\u001b[0m \u001b[38;5;66;03m# Execute callbacks (the only place they are executed; i.e. all entry points\u001b[39;00m\n\u001b[0;32m   2348\u001b[0m \u001b[38;5;66;03m# for callbacks will ultimately be directed to here for execution)\u001b[39;00m\n",
      "File \u001b[1;32mf:\\KTU\\Neuroninių tinklų metodai\\venv\\Lib\\site-packages\\tensorflow\\python\\checkpoint\\checkpoint.py:1330\u001b[0m, in \u001b[0;36mTrackableSaver.save\u001b[1;34m(self, file_prefix, checkpoint_number, session, options)\u001b[0m\n\u001b[0;32m   1327\u001b[0m   object_graph_tensor \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[0;32m   1329\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m tensor_util\u001b[38;5;241m.\u001b[39mis_tensor(file_prefix):\n\u001b[1;32m-> 1330\u001b[0m   \u001b[43mfile_io\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrecursive_create_dir\u001b[49m\u001b[43m(\u001b[49m\u001b[43mos\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mpath\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mdirname\u001b[49m\u001b[43m(\u001b[49m\u001b[43mfile_prefix\u001b[49m\u001b[43m)\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m   1332\u001b[0m save_path, new_feed_additions \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_save_cached_when_graph_building(\n\u001b[0;32m   1333\u001b[0m     file_prefix_tensor, object_graph_tensor, options)\n\u001b[0;32m   1335\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m new_feed_additions:\n",
      "File \u001b[1;32mf:\\KTU\\Neuroninių tinklų metodai\\venv\\Lib\\site-packages\\tensorflow\\python\\lib\\io\\file_io.py:498\u001b[0m, in \u001b[0;36mrecursive_create_dir\u001b[1;34m(dirname)\u001b[0m\n\u001b[0;32m    486\u001b[0m \u001b[38;5;129m@tf_export\u001b[39m(v1\u001b[38;5;241m=\u001b[39m[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mgfile.MakeDirs\u001b[39m\u001b[38;5;124m\"\u001b[39m])\n\u001b[0;32m    487\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mrecursive_create_dir\u001b[39m(dirname):\n\u001b[0;32m    488\u001b[0m \u001b[38;5;250m  \u001b[39m\u001b[38;5;124;03m\"\"\"Creates a directory and all parent/intermediate directories.\u001b[39;00m\n\u001b[0;32m    489\u001b[0m \n\u001b[0;32m    490\u001b[0m \u001b[38;5;124;03m  It succeeds if dirname already exists and is writable.\u001b[39;00m\n\u001b[1;32m   (...)\u001b[0m\n\u001b[0;32m    496\u001b[0m \u001b[38;5;124;03m    errors.OpError: If the operation fails.\u001b[39;00m\n\u001b[0;32m    497\u001b[0m \u001b[38;5;124;03m  \"\"\"\u001b[39;00m\n\u001b[1;32m--> 498\u001b[0m   \u001b[43mrecursive_create_dir_v2\u001b[49m\u001b[43m(\u001b[49m\u001b[43mdirname\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[1;32mf:\\KTU\\Neuroninių tinklų metodai\\venv\\Lib\\site-packages\\tensorflow\\python\\lib\\io\\file_io.py:513\u001b[0m, in \u001b[0;36mrecursive_create_dir_v2\u001b[1;34m(path)\u001b[0m\n\u001b[0;32m    501\u001b[0m \u001b[38;5;129m@tf_export\u001b[39m(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mio.gfile.makedirs\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m    502\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mrecursive_create_dir_v2\u001b[39m(path):\n\u001b[0;32m    503\u001b[0m \u001b[38;5;250m  \u001b[39m\u001b[38;5;124;03m\"\"\"Creates a directory and all parent/intermediate directories.\u001b[39;00m\n\u001b[0;32m    504\u001b[0m \n\u001b[0;32m    505\u001b[0m \u001b[38;5;124;03m  It succeeds if path already exists and is writable.\u001b[39;00m\n\u001b[1;32m   (...)\u001b[0m\n\u001b[0;32m    511\u001b[0m \u001b[38;5;124;03m    errors.OpError: If the operation fails.\u001b[39;00m\n\u001b[0;32m    512\u001b[0m \u001b[38;5;124;03m  \"\"\"\u001b[39;00m\n\u001b[1;32m--> 513\u001b[0m   \u001b[43m_pywrap_file_io\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mRecursivelyCreateDir\u001b[49m\u001b[43m(\u001b[49m\u001b[43mcompat\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mpath_to_bytes\u001b[49m\u001b[43m(\u001b[49m\u001b[43mpath\u001b[49m\u001b[43m)\u001b[49m\u001b[43m)\u001b[49m\n",
      "\u001b[1;31mFailedPreconditionError\u001b[0m: ./reference is not a directory"
     ]
    }
   ],
   "source": [
    "checkpoint.save(file_prefix = \"./reference/RNN_chpt\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Pakeiskite tokius parametrus kaip buferio dydis ir žodyno dydis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From f:\\KTU\\Neuroninių tinklų metodai\\venv\\Lib\\site-packages\\keras\\src\\backend.py:873: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From f:\\KTU\\Neuroninių tinklų metodai\\venv\\Lib\\site-packages\\keras\\src\\backend.py:873: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From f:\\KTU\\Neuroninių tinklų metodai\\venv\\Lib\\site-packages\\keras\\src\\utils\\tf_utils.py:492: The name tf.ragged.RaggedTensorValue is deprecated. Please use tf.compat.v1.ragged.RaggedTensorValue instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From f:\\KTU\\Neuroninių tinklų metodai\\venv\\Lib\\site-packages\\keras\\src\\utils\\tf_utils.py:492: The name tf.ragged.RaggedTensorValue is deprecated. Please use tf.compat.v1.ragged.RaggedTensorValue instead.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "BUFFER_SIZE = 1000\n",
    "BATCH_SIZE = 64\n",
    "VOCAB_SIZE = 100\n",
    "\n",
    "dataset, info = tfds.load('imdb_reviews', with_info=True, as_supervised=True)\n",
    "train_dataset, test_dataset = dataset['train'], dataset['test']\n",
    "\n",
    "train_dataset = train_dataset.shuffle(BUFFER_SIZE).batch(BATCH_SIZE).prefetch(tf.data.AUTOTUNE)\n",
    "test_dataset = test_dataset.batch(BATCH_SIZE).prefetch(tf.data.AUTOTUNE)\n",
    "\n",
    "encoder = tf.keras.layers.TextVectorization(max_tokens=VOCAB_SIZE)\n",
    "encoder.adapt(train_dataset.map(lambda text, label: text))\n",
    "\n",
    "model = tf.keras.Sequential([\n",
    "    encoder,\n",
    "    tf.keras.layers.Embedding(\n",
    "        input_dim=len(encoder.get_vocabulary()),\n",
    "        output_dim=64,\n",
    "        # Use masking to handle the variable sequence lengths\n",
    "        mask_zero=True\n",
    "    ),\n",
    "    tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(64)),\n",
    "    tf.keras.layers.Dense(64, activation='relu'),\n",
    "    tf.keras.layers.Dense(1)\n",
    "])\n",
    "\n",
    "model.compile(\n",
    "    loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),\n",
    "    optimizer=tf.keras.optimizers.Adam(1e-4),\n",
    "    metrics=['accuracy']\n",
    ")\n",
    "\n",
    "checkpoint = tf.train.Checkpoint(\n",
    "    model=model\n",
    ")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "WARNING:tensorflow:From f:\\KTU\\Neuroninių tinklų metodai\\venv\\Lib\\site-packages\\keras\\src\\engine\\base_layer_utils.py:384: The name tf.executing_eagerly_outside_functions is deprecated. Please use tf.compat.v1.executing_eagerly_outside_functions instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From f:\\KTU\\Neuroninių tinklų metodai\\venv\\Lib\\site-packages\\keras\\src\\engine\\base_layer_utils.py:384: The name tf.executing_eagerly_outside_functions is deprecated. Please use tf.compat.v1.executing_eagerly_outside_functions instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "391/391 [==============================] - 419s 1s/step - loss: 0.6884 - accuracy: 0.5036 - val_loss: 0.6677 - val_accuracy: 0.4917\n",
      "Epoch 2/10\n",
      "391/391 [==============================] - 483s 1s/step - loss: 0.6242 - accuracy: 0.5987 - val_loss: 0.6088 - val_accuracy: 0.6245\n",
      "Epoch 3/10\n",
      "391/391 [==============================] - 451s 1s/step - loss: 0.5704 - accuracy: 0.6875 - val_loss: 0.5618 - val_accuracy: 0.7161\n",
      "Epoch 4/10\n",
      "391/391 [==============================] - 461s 1s/step - loss: 0.5462 - accuracy: 0.7132 - val_loss: 0.5517 - val_accuracy: 0.7167\n",
      "Epoch 5/10\n",
      "391/391 [==============================] - 473s 1s/step - loss: 0.5398 - accuracy: 0.7162 - val_loss: 0.5479 - val_accuracy: 0.7078\n",
      "Epoch 6/10\n",
      "391/391 [==============================] - 471s 1s/step - loss: 0.5335 - accuracy: 0.7213 - val_loss: 0.5417 - val_accuracy: 0.7115\n",
      "Epoch 7/10\n",
      "391/391 [==============================] - 495s 1s/step - loss: 0.5311 - accuracy: 0.7222 - val_loss: 0.5438 - val_accuracy: 0.7120\n",
      "Epoch 8/10\n",
      "391/391 [==============================] - 629s 2s/step - loss: 0.5296 - accuracy: 0.7226 - val_loss: 0.5371 - val_accuracy: 0.7188\n",
      "Epoch 9/10\n",
      "391/391 [==============================] - 1174s 3s/step - loss: 0.5290 - accuracy: 0.7241 - val_loss: 0.5492 - val_accuracy: 0.7245\n",
      "Epoch 10/10\n",
      "391/391 [==============================] - 959s 2s/step - loss: 0.5254 - accuracy: 0.7247 - val_loss: 0.5409 - val_accuracy: 0.7021\n"
     ]
    }
   ],
   "source": [
    "history = model.fit(\n",
    "    train_dataset,\n",
    "    epochs=10,\n",
    "    validation_data=test_dataset,\n",
    "    validation_steps=30\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "ename": "RuntimeError",
     "evalue": "You must compile your model before training/testing. Use `model.compile(optimizer, loss)`.",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mRuntimeError\u001b[0m                              Traceback (most recent call last)",
      "Cell \u001b[1;32mIn[8], line 1\u001b[0m\n\u001b[1;32m----> 1\u001b[0m test_loss, test_acc \u001b[38;5;241m=\u001b[39m \u001b[43mmodel\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mevaluate\u001b[49m\u001b[43m(\u001b[49m\u001b[43mtest_dataset\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m      3\u001b[0m \u001b[38;5;28mprint\u001b[39m(\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mTest Loss:\u001b[39m\u001b[38;5;124m'\u001b[39m, test_loss)\n\u001b[0;32m      4\u001b[0m \u001b[38;5;28mprint\u001b[39m(\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mTest Accuracy:\u001b[39m\u001b[38;5;124m'\u001b[39m, test_acc)\n",
      "File \u001b[1;32mf:\\KTU\\Neuroninių tinklų metodai\\venv\\Lib\\site-packages\\keras\\src\\utils\\traceback_utils.py:70\u001b[0m, in \u001b[0;36mfilter_traceback.<locals>.error_handler\u001b[1;34m(*args, **kwargs)\u001b[0m\n\u001b[0;32m     67\u001b[0m     filtered_tb \u001b[38;5;241m=\u001b[39m _process_traceback_frames(e\u001b[38;5;241m.\u001b[39m__traceback__)\n\u001b[0;32m     68\u001b[0m     \u001b[38;5;66;03m# To get the full stack trace, call:\u001b[39;00m\n\u001b[0;32m     69\u001b[0m     \u001b[38;5;66;03m# `tf.debugging.disable_traceback_filtering()`\u001b[39;00m\n\u001b[1;32m---> 70\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m e\u001b[38;5;241m.\u001b[39mwith_traceback(filtered_tb) \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[0;32m     71\u001b[0m \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[0;32m     72\u001b[0m     \u001b[38;5;28;01mdel\u001b[39;00m filtered_tb\n",
      "File \u001b[1;32mf:\\KTU\\Neuroninių tinklų metodai\\venv\\Lib\\site-packages\\keras\\src\\engine\\training.py:3983\u001b[0m, in \u001b[0;36mModel._assert_compile_was_called\u001b[1;34m(self)\u001b[0m\n\u001b[0;32m   3977\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m_assert_compile_was_called\u001b[39m(\u001b[38;5;28mself\u001b[39m):\n\u001b[0;32m   3978\u001b[0m     \u001b[38;5;66;03m# Checks whether `compile` has been called. If it has been called,\u001b[39;00m\n\u001b[0;32m   3979\u001b[0m     \u001b[38;5;66;03m# then the optimizer is set. This is different from whether the\u001b[39;00m\n\u001b[0;32m   3980\u001b[0m     \u001b[38;5;66;03m# model is compiled\u001b[39;00m\n\u001b[0;32m   3981\u001b[0m     \u001b[38;5;66;03m# (i.e. whether the model is built and its inputs/outputs are set).\u001b[39;00m\n\u001b[0;32m   3982\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_is_compiled:\n\u001b[1;32m-> 3983\u001b[0m         \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mRuntimeError\u001b[39;00m(\n\u001b[0;32m   3984\u001b[0m             \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mYou must compile your model before \u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[0;32m   3985\u001b[0m             \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mtraining/testing. \u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[0;32m   3986\u001b[0m             \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mUse `model.compile(optimizer, loss)`.\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[0;32m   3987\u001b[0m         )\n",
      "\u001b[1;31mRuntimeError\u001b[0m: You must compile your model before training/testing. Use `model.compile(optimizer, loss)`."
     ]
    }
   ],
   "source": [
    "test_loss, test_acc = model.evaluate(test_dataset)\n",
    "\n",
    "print('Test Loss:', test_loss)\n",
    "print('Test Accuracy:', test_acc)\n",
    "\n",
    "plot_graphs(history, 'accuracy')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Pakeiskite Bidirectional sluoksnį į paprastą RNN ir palyginkite modelių tikslumą."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = tf.keras.Sequential([\n",
    "    encoder,\n",
    "    tf.keras.layers.Embedding(\n",
    "        input_dim=len(encoder.get_vocabulary()),\n",
    "        output_dim=64,\n",
    "        # Use masking to handle the variable sequence lengths\n",
    "        mask_zero=True\n",
    "    ),\n",
    "    tf.keras.layers.SimpleRNN(64),\n",
    "    tf.keras.layers.Dense(64, activation='relu'),\n",
    "    tf.keras.layers.Dense(1)\n",
    "])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
