chore(release): 1.0.1

fix: Resolves the app crash issue when there is no database.
closes #161, #162
2025-06-26 18:15:54 +00:00 · 2024-02-13 23:55:32 +06:00 · 2024-02-13 23:54:55 +06:00 · 2024-02-09 21:47:39 +06:00 · 2024-02-09 21:18:23 +06:00 · 2024-02-09 00:56:59 +06:00
56 changed files with 1976 additions and 306 deletions
--- a/.eslintrc.json
+++ b/.eslintrc.json
@@ -13,6 +13,7 @@
   "arrow-body-style":"off",
   "max-len": ["error", {"code": 150, "ignoreComments": true, "ignoreUrls": true}],
   "import/no-extraneous-dependencies": "off",
+   "no-unused-vars": "off",
   "import/extensions": [
      "error",
      "ignorePackages",
--- a/.sequelizerc
+++ b/.sequelizerc
@@ -0,0 +1,8 @@
+const path = require('path');
+
+module.exports = {
+  'config': path.resolve('database', 'config.js'),
+  'models-path': path.resolve('database', 'models'),
+  'seeders-path': path.resolve('database', 'seeders'),
+  'migrations-path': path.resolve('database', 'migrations')
+};
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -2,6 +2,31 @@

 All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.

+### [1.0.1](https://github.com/towfiqi/serpbear/compare/v1.0.0...v1.0.1) (2024-02-13)
+
+
+### Bug Fixes
+
+* Resolves the app crash issue when there is no database. ([e5dd411](https://github.com/towfiqi/serpbear/commit/e5dd411aa9aef58ebb226f2b793a2632ab9069a7)), closes [#161](https://github.com/towfiqi/serpbear/issues/161) [#162](https://github.com/towfiqi/serpbear/issues/162)
+
+## [1.0.0](https://github.com/towfiqi/serpbear/compare/v0.3.4...v1.0.0) (2024-02-09)
+
+
+### Features
+
+* Adds Serper.dev integration ([b4ad69b](https://github.com/towfiqi/serpbear/commit/b4ad69baaa0f865938f8b0eace6732a9e6b1b381)), closes [#138](https://github.com/towfiqi/serpbear/issues/138)
+* Adds the ability for city level scraping for scapers that allow it. ([3719f21](https://github.com/towfiqi/serpbear/commit/3719f21d98d173219cef5656579fa0e5340ccdbf)), closes [#139](https://github.com/towfiqi/serpbear/issues/139) [#151](https://github.com/towfiqi/serpbear/issues/151)
+* adds the ability to add url as a domain. ([3c2a1b8](https://github.com/towfiqi/serpbear/commit/3c2a1b8a5b8a2a4a2179a5031582f8202c2e494a)), closes [#53](https://github.com/towfiqi/serpbear/issues/53) [#90](https://github.com/towfiqi/serpbear/issues/90) [#119](https://github.com/towfiqi/serpbear/issues/119)
+* Adds the Ability to set Search Console Property type via Domain Settings. ([b2e97b2](https://github.com/towfiqi/serpbear/commit/b2e97b2ebec380f0edf7ddc0640c2126eff006ac)), closes [#50](https://github.com/towfiqi/serpbear/issues/50)
+* Adds the ability to setup Search Console through the UI. ([f04b10c](https://github.com/towfiqi/serpbear/commit/f04b10cf6b065e3023965112a60e0aa702212a4b)), closes [#59](https://github.com/towfiqi/serpbear/issues/59) [#146](https://github.com/towfiqi/serpbear/issues/146)
+* Adds ValueSerp Integration. ([1041cb3](https://github.com/towfiqi/serpbear/commit/1041cb3c0bb69e9034696624e03433be28e83ac6)), closes [#105](https://github.com/towfiqi/serpbear/issues/105) [#106](https://github.com/towfiqi/serpbear/issues/106)
+
+
+### Bug Fixes
+
+* Resolves Keywords filter crashing issue. ([633ab2c](https://github.com/towfiqi/serpbear/commit/633ab2c467be5b7b86d4547ae0c59034e595a42d))
+* Resolves missing Keyword Loading Spinner issue. ([dbf540c](https://github.com/towfiqi/serpbear/commit/dbf540cfdb16ddb02c9d26618e3680d34799f57f))
+
 ### [0.3.4](https://github.com/towfiqi/serpbear/compare/v0.3.3...v0.3.4) (2024-01-15)


--- a/6
+++ b/6
@@ -30,13 +30,17 @@ COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static
 # setup the cron
 COPY --from=builder --chown=nextjs:nodejs /app/cron.js ./
 COPY --from=builder --chown=nextjs:nodejs /app/email ./email
+COPY --from=builder --chown=nextjs:nodejs /app/database ./database
+COPY --from=builder --chown=nextjs:nodejs /app/.sequelizerc ./.sequelizerc
+COPY --from=builder --chown=nextjs:nodejs /app/entrypoint.sh ./entrypoint.sh
 RUN rm package.json
 RUN npm init -y 
-RUN npm i cryptr dotenv croner @googleapis/searchconsole
+RUN npm i cryptr dotenv croner @googleapis/searchconsole sequelize-cli
 RUN npm i -g concurrently

 USER nextjs

 EXPOSE 3000

+ENTRYPOINT ["/app/entrypoint.sh"]
 CMD ["concurrently","node server.js", "node cron.js"]
--- a/README.md
+++ b/README.md
@@ -42,6 +42,7 @@ The App uses third party website scrapers like ScrapingAnt, ScrapingRobot, Searc
 | serpapi.com | From $50/mo** | From 5,000/mo** | Yes |
 | spaceserp.com | $59/lifetime | 15,000/mo | Yes |
 | SearchApi.io | From $40/mo | From 10,000/mo | Yes |
+| valueserp.com | Pay As You Go | $2.50/1000 req | No |

 (*) Free upto a limit. If you are using ScrapingAnt you can lookup 10,000 times per month for free.
 (**) Free up to 100 per month. Paid from 5,000 to 10,000,000+ per month.
--- a/components/common/Icon.tsx
+++ b/components/common/Icon.tsx
@@ -253,6 +253,27 @@ const Icon = ({ type, color = 'currentColor', size = 16, title = '', classes = '
                  <path d="M15.75 9h3v2.25h-3z" fill={color} />
               </svg>
         }
+         {type === 'email'
+            && <svg {...xmlnsProps} width={size} viewBox="0 0 24 24">
+                  <path fill={color} d="M22 6c0-1.1-.9-2-2-2H4c-1.1 0-2 .9-2 2v12c0 1.1.9 2 2 2h16c1.1 0 2-.9 2-2zm-2 0l-8 5l-8-5zm0 12H4V8l8 5l8-5z" />
+               </svg>
+         }
+         {type === 'scraper'
+            && <svg {...xmlnsProps} width={size} viewBox="0 0 16 16">
+               <path fill={color} d="M1 3.5A2.5 2.5 0 0 1 3.5 1h7A2.5 2.5 0 0 1 13 3.5v1.53a4.538 4.538 0 0 0-1-.004V5H2v5.5A1.5 1.5 0 0 0 3.5 12h2.954l-.72.72a2.52 2.52 0 0 0-.242.28H3.5A2.5 2.5 0 0 1 1 10.5zm7.931 3.224l-.577-.578a.5.5 0 1 0-.708.708l.745.744c.144-.306.324-.6.54-.874M2 4h10v-.5A1.5 1.5 0 0 0 10.5 2h-7A1.5 1.5 0 0 0 2 3.5zm4.354 2.854a.5.5 0 1 0-.708-.708l-2 2a.5.5 0 0 0 0 .708l2 2a.5.5 0 0 0 .708-.708L4.707 8.5zm6.538-.83c.366.042.471.48.21.742l-.975.975a1.507 1.507 0 1 0 2.132 2.132l.975-.975c.261-.261.7-.156.742.21a3.518 3.518 0 0 1-4.676 3.723l-2.726 2.727a1.507 1.507 0 1 1-2.132-2.132L9.168 10.7a3.518 3.518 0 0 1 3.724-4.676" />
+            </svg>
+         }
+         {type === 'city'
+            && <svg {...xmlnsProps} width={size} viewBox="0 0 48 48">
+                  <g fill="none">
+                     <path stroke={color} strokeLinecap="round" strokeLinejoin="round" strokeWidth={4} d="M4 42h40"></path>
+                     <rect width={8} height={16} x={8} y={26} stroke={color} strokeLinejoin="round" strokeWidth={4} rx={2}></rect>
+                     <path stroke={color} strokeLinecap="square" strokeLinejoin="round" strokeWidth={4} d="M12 34h1"></path>
+                     <rect width={24} height={38} x={16} y={4} stroke={color} strokeLinejoin="round" strokeWidth={4} rx={2}></rect>
+                     <path fill={color} d="M22 10h4v4h-4zm8 0h4v4h-4zm-8 7h4v4h-4zm8 0h4v4h-4zm0 7h4v4h-4zm0 7h4v4h-4z"></path>
+                  </g>
+               </svg>
+         }
       </span>
   );
 };
--- a/components/common/InputField.tsx
+++ b/components/common/InputField.tsx
@@ -0,0 +1,28 @@
+type InputFieldProps = {
+   label: string;
+   value: string;
+   onChange: Function;
+   placeholder?: string;
+   classNames?: string;
+   hasError?: boolean;
+}
+
+const InputField = ({ label = '', value = '', placeholder = '', onChange, hasError = false }: InputFieldProps) => {
+   const labelStyle = 'mb-2 font-semibold inline-block text-sm text-gray-700 capitalize';
+   return (
+      <div className="field--input w-full relative flex justify-between items-center">
+         <label className={labelStyle}>{label}</label>
+         <input
+            className={`p-2 border border-gray-200 rounded focus:outline-none w-[210px]
+             focus:border-blue-200 ${hasError ? ' border-red-400 focus:border-red-400' : ''} `}
+            type={'text'}
+            value={value}
+            onChange={(event) => onChange(event.target.value)}
+            autoComplete="off"
+            placeholder={placeholder}
+         />
+      </div>
+   );
+};
+
+export default InputField;
--- a/components/common/Modal.tsx
+++ b/components/common/Modal.tsx
@@ -6,10 +6,11 @@ type ModalProps = {
   children: React.ReactNode,
   width?: string,
   title?: string,
+   verticalCenter?: boolean,
   closeModal: Function,
 }

-const Modal = ({ children, width = '1/2', closeModal, title }:ModalProps) => {
+const Modal = ({ children, width = '1/2', closeModal, title, verticalCenter = false }:ModalProps) => {
   useOnKey('Escape', closeModal);

   const closeOnBGClick = (e:React.SyntheticEvent) => {
@@ -21,8 +22,9 @@ const Modal = ({ children, width = '1/2', closeModal, title }:ModalProps) => {
   return (
      <div className='modal fixed top-0 left-0 bg-white/[.7] w-full h-screen z-50' onClick={closeOnBGClick}>
         <div
-         className={`modal__content max-w-[340px] absolute top-1/4 left-0 right-0 ml-auto mr-auto w-${width} 
-         lg:max-w-md bg-white shadow-md rounded-md p-5 border-t-[1px] border-gray-100 text-base`}>
+         className={`modal__content max-w-[340px] absolute left-0 right-0 ml-auto mr-auto w-${width} 
+         lg:max-w-md bg-white shadow-md rounded-md p-5 border-t-[1px] border-gray-100 text-base 
+         ${verticalCenter ? ' top-1/2 translate-y-[-50%]' : 'top-1/4'}`}>
            {title && <h3 className=' font-semibold mb-3'>{title}</h3>}
            <button
            className='modal-close absolute right-2 top-2 p-2 cursor-pointer text-gray-400 transition-all
--- a/components/common/SecretField.tsx
+++ b/components/common/SecretField.tsx
@@ -14,15 +14,15 @@ const SecretField = ({ label = '', value = '', placeholder = '', onChange, hasEr
   const [showValue, setShowValue] = useState(false);
   const labelStyle = 'mb-2 font-semibold inline-block text-sm text-gray-700 capitalize';
   return (
-      <div className="settings__section__secret mb-5 relative">
+      <div className="settings__section__secret mb-5 relative flex justify-between items-center">
         <label className={labelStyle}>{label}</label>
         <span
-         className="absolute top-8 right-0 px-2 py-1 cursor-pointer text-gray-400 select-none"
+         className="absolute top-1 right-0 px-2 py-1 cursor-pointer text-gray-400 select-none"
         onClick={() => setShowValue(!showValue)}>
            <Icon type={showValue ? 'eye-closed' : 'eye'} size={18} />
         </span>
         <input
-            className={`w-full p-2 border border-gray-200 rounded mb-3 focus:outline-none 
+            className={`w-[210px] p-2 border border-gray-200 rounded focus:outline-none 
             focus:border-blue-200 ${hasError ? ' border-red-400 focus:border-red-400' : ''} `}
            type={showValue ? 'text' : 'password'}
            value={value}
--- a/components/common/SelectField.tsx
+++ b/components/common/SelectField.tsx
@@ -9,6 +9,7 @@ type SelectFieldProps = {
   defaultLabel: string,
   options: SelectionOption[],
   selected: string[],
+   label?: string,
   multiple?: boolean,
   updateField: Function,
   minWidth?: number,
@@ -28,6 +29,7 @@ const SelectField = (props: SelectFieldProps) => {
      maxHeight = 96,
      rounded = 'rounded-3xl',
      flags = false,
+      label = '',
      emptyMsg = '' } = props;

   const [showOptions, setShowOptions] = useState<boolean>(false);
@@ -66,12 +68,13 @@ const SelectField = (props: SelectFieldProps) => {
   };

   return (
-       <div className="select font-semibold text-gray-500">
+       <div className="select font-semibold text-gray-500 relative flex justify-between items-center">
+         {label && <label className='mb-2 font-semibold inline-block text-sm text-gray-700 capitalize'>{label}</label>}
         <div
-         className={`selected flex border ${rounded} p-1.5 px-4 cursor-pointer select-none w-[180px] min-w-[${minWidth}px] 
+         className={`selected flex border ${rounded} p-1.5 px-4 cursor-pointer select-none w-[210px] min-w-[${minWidth}px] 
         ${showOptions ? 'border-indigo-200' : ''}`}
         onClick={() => setShowOptions(!showOptions)}>
-            <span className={`w-[${minWidth - 30}px] inline-block truncate mr-2 capitalize`}>
+            <span className={'w-full inline-block truncate mr-2 capitalize'}>
               {selected.length > 0 ? (selectedLabels.slice(0, 2).join(', ')) : defaultLabel}
            </span>
            {multiple && selected.length > 2
@@ -80,7 +83,7 @@ const SelectField = (props: SelectFieldProps) => {
         </div>
         {showOptions && (
            <div
-            className={`select_list mt-1 border absolute min-w-[${minWidth}px] 
+            className={`select_list mt-1 border absolute min-w-[${minWidth}px] top-[30px] right-0 w-[210px]
            ${rounded === 'rounded-3xl' ? 'rounded-lg' : rounded} max-h-${maxHeight} bg-white z-50 overflow-y-auto styled-scrollbar`}>
               {options.length > 20 && (
                  <div className=''>
--- a/components/common/ToggleField.tsx
+++ b/components/common/ToggleField.tsx
@@ -0,0 +1,32 @@
+type ToggleFieldProps = {
+   label: string;
+   value: string;
+   onChange: (bool:boolean) => void ;
+   classNames?: string;
+}
+
+const ToggleField = ({ label = '', value = '', onChange, classNames = '' }: ToggleFieldProps) => {
+   return (
+      <div className={`field--toggle w-full relative ${classNames}`}>
+         <label className="relative inline-flex items-center cursor-pointer w-full justify-between">
+            <span className="text-sm font-medium text-gray-900 dark:text-gray-300 w-56">{label}</span>
+            <input
+            type="checkbox"
+            value={value}
+            checked={!!value}
+            className="sr-only peer"
+            onChange={() => onChange(!value)}
+            />
+            <div className="relative rounded-3xl w-9 h-5 bg-gray-200 peer-focus:outline-none peer-focus:ring-4
+            peer-focus:ring-blue-300 dark:peer-focus:ring-blue-800rounded-full peer dark:bg-gray-700
+            peer-checked:after:translate-x-full peer-checked:after:border-white after:content-['']
+            after:absolute after:top-[2px] after:left-[2px] after:bg-white  after:border-gray-300
+            after:border after:rounded-full after:h-4 after:w-4
+            after:transition-all dark:border-gray-600 peer-checked:bg-blue-600"></div>
+
+         </label>
+      </div>
+   );
+};
+
+export default ToggleField;
--- a/components/domains/AddDomain.tsx
+++ b/components/domains/AddDomain.tsx
@@ -1,7 +1,7 @@
 import React, { useState } from 'react';
 import Modal from '../common/Modal';
 import { useAddDomain } from '../../services/domains';
-import { isValidDomain } from '../../utils/client/validators';
+import { isValidUrl } from '../../utils/client/validators';

 type AddDomainProps = {
   domains: DomainType[],
@@ -16,24 +16,30 @@ const AddDomain = ({ closeModal, domains = [] }: AddDomainProps) => {
   const addDomain = () => {
      setNewDomainError('');
      const existingDomains = domains.map((d) => d.domain);
-      const insertedDomains = newDomain.split('\n');
+      const insertedURLs = newDomain.split('\n');
      const domainsTobeAdded:string[] = [];
      const invalidDomains:string[] = [];
-      insertedDomains.forEach((dom) => {
-        const domain = dom.trim();
-        if (isValidDomain(domain)) {
-         if (!existingDomains.includes(domain)) {
-            domainsTobeAdded.push(domain);
+      insertedURLs.forEach((url) => {
+        const theURL = url.trim();
+        if (isValidUrl(theURL)) {
+         const domURL = new URL(theURL);
+         const isDomain = domURL.pathname === '/';
+         if (isDomain && !existingDomains.includes(domURL.host)) {
+            domainsTobeAdded.push(domURL.host);
+         }
+         if (!isDomain && !existingDomains.includes(domURL.href)) {
+            const cleanedURL = domURL.href.replace('https://', '').replace('http://', '').replace(/^\/+|\/+$/g, '');
+            domainsTobeAdded.push(cleanedURL);
         }
        } else {
-         invalidDomains.push(domain);
+         invalidDomains.push(theURL);
        }
      });
      if (invalidDomains.length > 0) {
-         setNewDomainError(`Please Insert Valid Domain names. Invalid Domains: ${invalidDomains.join(', ')}`);
+         setNewDomainError(`Please Insert Valid Domain URL. Invalid URLs: ${invalidDomains.join(', ')}`);
      } else if (domainsTobeAdded.length > 0) {
-         // TODO: Domain Action
-          addMutate(domainsTobeAdded);
+            console.log('domainsTobeAdded :', domainsTobeAdded);
+         addMutate(domainsTobeAdded);
      }
   };

@@ -45,11 +51,11 @@ const AddDomain = ({ closeModal, domains = [] }: AddDomainProps) => {
   return (
      <Modal closeModal={() => { closeModal(false); }} title={'Add New Domain'}>
         <div data-testid="adddomain_modal">
-            <h4 className='text-sm mt-4'>Domain Names</h4>
+            <h4 className='text-sm mt-4'>Domain URL</h4>
            <textarea
               className={`w-full h-40 border rounded border-gray-200 p-4 outline-none
                focus:border-indigo-300 ${newDomainError ? ' border-red-400 focus:border-red-400' : ''}`}
-               placeholder="Type or Paste Domains here. Insert Each Domain in a New line."
+               placeholder="Type or Paste URLs here. Insert Each URL in a New line."
               value={newDomain}
               autoFocus={true}
               onChange={handleDomainInput}>
--- a/components/domains/DomainItem.tsx
+++ b/components/domains/DomainItem.tsx
@@ -38,7 +38,7 @@ const DomainItem = ({ domain, selected, isConsoleIntegrated = false, thumb, upda
                  />
               </div>
               <div className="domain_details flex-1">
-                  <h3 className='font-semibold text-base mb-2'>{domain.domain}</h3>
+                  <h3 className='font-semibold text-base mb-2 max-w-[200px] text-ellipsis overflow-hidden' title={domain.domain}>{domain.domain}</h3>
                 {keywordsUpdated && (
                  <span className=' text-gray-600 text-xs'>
                     Updated <TimeAgo title={dayjs(keywordsUpdated).format('DD-MMM-YYYY, hh:mm:ss A')} date={keywordsUpdated} />
--- a/components/domains/DomainSettings.tsx
+++ b/components/domains/DomainSettings.tsx
@@ -2,7 +2,9 @@ import { useRouter } from 'next/router';
 import { useState } from 'react';
 import Icon from '../common/Icon';
 import Modal from '../common/Modal';
-import { useDeleteDomain, useUpdateDomain } from '../../services/domains';
+import { useDeleteDomain, useFetchDomain, useUpdateDomain } from '../../services/domains';
+import InputField from '../common/InputField';
+import SelectField from '../common/SelectField';

 type DomainSettingsProps = {
   domain:DomainType|false,
@@ -16,25 +18,27 @@ type DomainSettingsError = {

 const DomainSettings = ({ domain, closeModal }: DomainSettingsProps) => {
   const router = useRouter();
+   const [currentTab, setCurrentTab] = useState<'notification'|'searchconsole'>('notification');
   const [showRemoveDomain, setShowRemoveDomain] = useState<boolean>(false);
   const [settingsError, setSettingsError] = useState<DomainSettingsError>({ type: '', msg: '' });
   const [domainSettings, setDomainSettings] = useState<DomainSettings>(() => ({
      notification_interval: domain && domain.notification_interval ? domain.notification_interval : 'never',
      notification_emails: domain && domain.notification_emails ? domain.notification_emails : '',
+      search_console: domain && domain.search_console ? JSON.parse(domain.search_console) : {
+         property_type: 'domain', url: '', client_email: '', private_key: '',
+      },
   }));

-   const { mutate: updateMutate } = useUpdateDomain(() => closeModal(false));
-   const { mutate: deleteMutate } = useDeleteDomain(() => {
-      closeModal(false);
-      router.push('/domains');
+   const { mutate: updateMutate, error: domainUpdateError, isLoading: isUpdating } = useUpdateDomain(() => closeModal(false));
+   const { mutate: deleteMutate } = useDeleteDomain(() => { closeModal(false); router.push('/domains'); });
+
+   // Get the Full Domain Data along with the Search Console API Data.
+   useFetchDomain(router, domain && domain.domain ? domain.domain : '', (domainObj:DomainType) => {
+      const currentSearchConsoleSettings = domainObj.search_console && JSON.parse(domainObj.search_console);
+      setDomainSettings({ ...domainSettings, search_console: currentSearchConsoleSettings || domainSettings.search_console });
   });

-   const updateNotiEmails = (event:React.FormEvent<HTMLInputElement>) => {
-      setDomainSettings({ ...domainSettings, notification_emails: event.currentTarget.value });
-   };
-
   const updateDomain = () => {
-      console.log('Domain: ');
      let error: DomainSettingsError | null = null;
      if (domainSettings.notification_emails) {
         const notification_emails = domainSettings.notification_emails.split(',');
@@ -55,24 +59,103 @@ const DomainSettings = ({ domain, closeModal }: DomainSettingsProps) => {
         }
   };

+   const tabStyle = `inline-block px-4 py-2 rounded-md mr-3 cursor-pointer text-sm select-none z-10
+                     text-gray-600 border border-b-0 relative top-[1px] rounded-b-none`;
   return (
      <div>
-         <Modal closeModal={() => closeModal(false)} title={'Domain Settings'} width="[500px]">
+         <Modal closeModal={() => closeModal(false)} title={'Domain Settings'} width="[500px]" verticalCenter={currentTab === 'searchconsole'} >
            <div data-testid="domain_settings" className=" text-sm">
-               <div className="mb-6 flex justify-between items-center">
-                  <h4>Notification Emails
-                     {settingsError.type === 'email' && <span className="text-red-500 font-semibold ml-2">{settingsError.msg}</span>}
-                  </h4>
-                  <input
-                     className={`border w-46 text-sm transition-all rounded p-1.5 px-4 outline-none ring-0 
-                     ${settingsError.type === 'email' ? ' border-red-300' : ''}`}
-                     type="text"
-                     placeholder='Your Emails'
-                     onChange={updateNotiEmails}
-                     value={domainSettings.notification_emails || ''}
-                  />
+               <div className=' mt-3 mb-5 border  border-slate-200 px-2 py-4 pb-0
+               relative left-[-20px] w-[calc(100%+40px)] border-l-0 border-r-0 bg-[#f8f9ff]'>
+                  <ul>
+                     <li
+                     className={`${tabStyle} ${currentTab === 'notification' ? ' bg-white text-blue-600 border-slate-200' : 'border-transparent'} `}
+                     onClick={() => setCurrentTab('notification')}>
+                       <Icon type='email' /> Notification
+                     </li>
+                     <li
+                     className={`${tabStyle} ${currentTab === 'searchconsole' ? ' bg-white text-blue-600 border-slate-200' : 'border-transparent'}`}
+                     onClick={() => setCurrentTab('searchconsole')}>
+                        <Icon type='google' /> Search Console
+                     </li>
+                  </ul>
               </div>
+
+               <div>
+                  {currentTab === 'notification' && (
+                     <div className="mb-4 flex justify-between items-center w-full">
+                        <InputField
+                        label='Notification Emails'
+                        onChange={(emails:string) => setDomainSettings({ ...domainSettings, notification_emails: emails })}
+                        value={domainSettings.notification_emails || ''}
+                        placeholder='Your Emails'
+                        />
+                     </div>
+                  )}
+                  {currentTab === 'searchconsole' && (
+                     <>
+                        <div className="mb-4 flex justify-between items-center w-full">
+                           <label className='mb-2 font-semibold inline-block text-sm text-gray-700 capitalize'>Property Type</label>
+                           <SelectField
+                           options={[{ label: 'Domain', value: 'domain' }, { label: 'URL', value: 'url' }]}
+                           selected={[domainSettings.search_console?.property_type || 'domain']}
+                           defaultLabel="Select Search Console Property Type"
+                           updateField={(updated:['domain'|'url']) => setDomainSettings({
+                              ...domainSettings,
+                              search_console: { ...(domainSettings.search_console as DomainSearchConsole), property_type: updated[0] || 'domain' },
+                           })}
+                           multiple={false}
+                           rounded={'rounded'}
+                           />
+                        </div>
+                        {domainSettings?.search_console?.property_type === 'url' && (
+                           <div className="mb-4 flex justify-between items-center w-full">
+                              <InputField
+                              label='Property URL (Required)'
+                              onChange={(url:string) => setDomainSettings({
+                                 ...domainSettings,
+                                 search_console: { ...(domainSettings.search_console as DomainSearchConsole), url },
+                              })}
+                              value={domainSettings?.search_console?.url || ''}
+                              placeholder='Search Console Property URL. eg: https://mywebsite.com/'
+                              />
+                           </div>
+                        )}
+                        <div className="mb-4 flex justify-between items-center w-full">
+                           <InputField
+                           label='Search Console Client Email'
+                           onChange={(client_email:string) => setDomainSettings({
+                              ...domainSettings,
+                              search_console: { ...(domainSettings.search_console as DomainSearchConsole), client_email },
+                           })}
+                           value={domainSettings?.search_console?.client_email || ''}
+                           placeholder='myapp@appspot.gserviceaccount.com'
+                           />
+                        </div>
+                        <div className="mb-4 flex flex-col justify-between items-center w-full">
+                           <label className='mb-2 font-semibold block text-sm text-gray-700 capitalize w-full'>Search Console Private Key</label>
+                           <textarea
+                              className={`w-full p-2 border border-gray-200 rounded mb-3 text-xs 
+                              focus:outline-none h-[100px] focus:border-blue-200`}
+                              value={domainSettings?.search_console?.private_key || ''}
+                              placeholder={'-----BEGIN PRIVATE KEY-----/ssssaswdkihad....'}
+                              onChange={(event) => setDomainSettings({
+                                 ...domainSettings,
+                                 search_console: { ...(domainSettings.search_console as DomainSearchConsole), private_key: event.target.value },
+                              })}
+                           />
+                        </div>
+                     </>
+                  )}
+               </div>
+               {!isUpdating && (domainUpdateError as Error)?.message && (
+                  <div className='w-full mt-4 p-3 text-sm bg-red-50 text-red-700'>{(domainUpdateError as Error).message}</div>
+               )}
+               {!isUpdating && settingsError?.msg && (
+                  <div className='w-full mt-4 p-3 text-sm bg-red-50 text-red-700'>{settingsError.msg}</div>
+               )}
            </div>
+
            <div className="flex justify-between border-t-[1px] border-gray-100 mt-8 pt-4 pb-0">
               <button
               className="text-sm font-semibold text-red-500"
@@ -80,9 +163,9 @@ const DomainSettings = ({ domain, closeModal }: DomainSettingsProps) => {
                  <Icon type="trash" /> Remove Domain
               </button>
               <button
-               className='text-sm font-semibold py-2 px-5 rounded cursor-pointer bg-blue-700 text-white'
-               onClick={() => updateDomain()}>
-                  Update Settings
+               className={`text-sm font-semibold py-2 px-5 rounded cursor-pointer bg-blue-700 text-white ${isUpdating ? 'cursor-not-allowed' : ''}`}
+               onClick={() => !isUpdating && updateDomain()}>
+                  {isUpdating && <Icon type='loading' />} Update Settings
               </button>
            </div>
         </Modal>
--- a/components/insight/Insight.tsx
+++ b/components/insight/Insight.tsx
@@ -117,7 +117,7 @@ const SCInsight = ({ insight, isLoading = true, isConsoleIntegrated = true, doma
                     )}
                     {!isConsoleIntegrated && (
                        <p className=' p-9 pt-[10%] text-center text-gray-500'>
-                        Google Search has not been Integrated yet. Please follow <a className='text-indigo-600 underline' href='https://docs.serpbear.com/miscellaneous/integrate-google-search-console' target="_blank" rel='noreferrer'>These Steps</a> to integrate Google Search Data for this Domain.
+                        Google Search Console has not been Integrated yet. Please follow <a className='text-indigo-600 underline' href='https://docs.serpbear.com/miscellaneous/integrate-google-search-console' target="_blank" rel='noreferrer'>These Steps</a> to integrate Google Search Data for this Domain.
                        </p>
                     )}
                  </div>
--- a/components/keywords/AddKeywords.tsx
+++ b/components/keywords/AddKeywords.tsx
@@ -7,6 +7,8 @@ import { useAddKeywords } from '../../services/keywords';

 type AddKeywordsProps = {
   keywords: KeywordType[],
+   scraperName: string,
+   allowsCity: boolean,
   closeModal: Function,
   domain: string
 }
@@ -17,9 +19,10 @@ type KeywordsInput = {
   country: string,
   domain: string,
   tags: string,
+   city?:string,
 }

-const AddKeywords = ({ closeModal, domain, keywords }: AddKeywordsProps) => {
+const AddKeywords = ({ closeModal, domain, keywords, scraperName = '', allowsCity = false }: AddKeywordsProps) => {
   const [error, setError] = useState<string>('');
   const defCountry = localStorage.getItem('default_country') || 'US';
   const [newKeywordsData, setNewKeywordsData] = useState<KeywordsInput>({ keywords: '', device: 'desktop', country: defCountry, domain, tags: '' });
@@ -29,14 +32,16 @@ const AddKeywords = ({ closeModal, domain, keywords }: AddKeywordsProps) => {
   const addKeywords = () => {
      if (newKeywordsData.keywords) {
         const keywordsArray = [...new Set(newKeywordsData.keywords.split('\n').map((item) => item.trim()).filter((item) => !!item))];
-         const currentKeywords = keywords.map((k) => `${k.keyword}-${k.device}-${k.country}`);
-         const keywordExist = keywordsArray.filter((k) => currentKeywords.includes(`${k}-${newKeywordsData.device}-${newKeywordsData.country}`));
+         const currentKeywords = keywords.map((k) => `${k.keyword}-${k.device}-${k.country}${k.city ? `-${k.city}` : ''}`);
+         const keywordExist = keywordsArray.filter((k) => currentKeywords.includes(
+            `${k}-${newKeywordsData.device}-${newKeywordsData.country}${newKeywordsData.city ? `-${newKeywordsData.city}` : ''}`,
+         ));
         if (keywordExist.length > 0) {
            setError(`Keywords ${keywordExist.join(',')} already Exist`);
            setTimeout(() => { setError(''); }, 3000);
         } else {
-            const { device, country, domain: kDomain, tags } = newKeywordsData;
-            const newKeywordsArray = keywordsArray.map((nItem) => ({ keyword: nItem, device, country, domain: kDomain, tags }));
+            const { device, country, domain: kDomain, tags, city } = newKeywordsData;
+            const newKeywordsArray = keywordsArray.map((nItem) => ({ keyword: nItem, device, country, domain: kDomain, tags, city }));
            addMutate(newKeywordsArray);
         }
      } else {
@@ -85,17 +90,28 @@ const AddKeywords = ({ closeModal, domain, keywords }: AddKeywordsProps) => {
                        ><Icon type='mobile' /> <i className='not-italic hidden lg:inline-block'>Mobile</i></li>
                  </ul>
               </div>
-
               <div className='relative'>
                  {/* TODO:  Insert Existing Tags as Suggestions */}
                  <input
                     className='w-full border rounded border-gray-200 py-2 px-4 pl-8 outline-none focus:border-indigo-300'
-                     placeholder='Insert Tags'
+                     placeholder='Insert Tags (Optional)'
                     value={newKeywordsData.tags}
                     onChange={(e) => setNewKeywordsData({ ...newKeywordsData, tags: e.target.value })}
                  />
                  <span className='absolute text-gray-400 top-2 left-2'><Icon type="tags" size={16} /></span>
               </div>
+               <div className='relative mt-2'>
+                  <input
+                     className={`w-full border rounded border-gray-200 py-2 px-4 pl-8 
+                     outline-none focus:border-indigo-300 ${!allowsCity ? ' cursor-not-allowed' : ''} `}
+                     disabled={!allowsCity}
+                     title={!allowsCity ? `Your scraper ${scraperName} doesn't have city level scraping feature.` : ''}
+                     placeholder={`City (Optional)${!allowsCity ? `. Not avaialable for ${scraperName}.` : ''}`}
+                     value={newKeywordsData.city}
+                     onChange={(e) => setNewKeywordsData({ ...newKeywordsData, city: e.target.value })}
+                  />
+                  <span className='absolute text-gray-400 top-2 left-2'><Icon type="city" size={16} /></span>
+               </div>
            </div>
            {error && <div className='w-full mt-4 p-3 text-sm bg-red-50 text-red-700'>{error}</div>}
            <div className='mt-6 text-right text-sm font-semibold flex justify-between'>
--- a/components/keywords/Keyword.tsx
+++ b/components/keywords/Keyword.tsx
@@ -40,7 +40,7 @@ const Keyword = (props: KeywordProps) => {
      scDataType = 'threeDays',
   } = props;
   const {
-      keyword, domain, ID, position, url = '', lastUpdated, country, sticky, history = {}, updating = false, lastUpdateError = false,
+      keyword, domain, ID, city, position, url = '', lastUpdated, country, sticky, history = {}, updating = false, lastUpdateError = false,
   } = keywordData;
   const [showOptions, setShowOptions] = useState(false);
   const [showPositionError, setPositionError] = useState(false);
@@ -85,12 +85,12 @@ const Keyword = (props: KeywordProps) => {

   return (
      <div
-      key={keyword}
+      key={keyword + ID}
      style={style}
      className={`keyword relative py-5 px-4 text-gray-600 border-b-[1px] border-gray-200 lg:py-4 lg:px-6 lg:border-0 
      lg:flex lg:justify-between lg:items-center ${selected ? ' bg-indigo-50 keyword--selected' : ''} ${lastItem ? 'border-b-0' : ''}`}>

-         <div className=' w-3/4 lg:flex-1 lg:basis-20 lg:w-auto font-semibold cursor-pointer'>
+         <div className=' w-3/4 font-semibold cursor-pointer lg:flex-1 lg:basis-20 lg:w-auto lg:flex lg:items-center'>
            <button
               className={`p-0 mr-2 leading-[0px] inline-block rounded-sm pt-0 px-[1px] pb-[3px] border 
               ${selected ? ' bg-blue-700 border-blue-700 text-white' : 'text-transparent'}`}
@@ -99,9 +99,10 @@ const Keyword = (props: KeywordProps) => {
                  <Icon type="check" size={10} />
            </button>
            <a
-            className='py-2 hover:text-blue-600'
+            className='py-2 hover:text-blue-600 lg:flex lg:items-center lg:w-full'
            onClick={() => showKeywordDetails()}>
-               <span className={`fflag fflag-${country} w-[18px] h-[12px] mr-2`} title={countries[country][0]} />{keyword}
+               <span className={`fflag fflag-${country} w-[18px] h-[12px] mr-2`} title={countries[country][0]} />
+               <span className=' text-ellipsis overflow-hidden whitespace-nowrap w-[calc(100%-30px)]'>{keyword}{city ? ` (${city})` : ''}</span>
            </a>
            {sticky && <button className='ml-2 relative top-[2px]' title='Favorite'><Icon type="star-filled" size={16} color="#fbd346" /></button>}
            {lastUpdateError && lastUpdateError.date
@@ -114,7 +115,7 @@ const Keyword = (props: KeywordProps) => {
         <div
         className={`keyword_position absolute bg-[#f8f9ff] w-fit min-w-[50px] h-12 p-2 text-base mt-[-20px] rounded right-5 lg:relative
          lg:bg-transparent lg:w-auto lg:h-auto lg:mt-0 lg:p-0 lg:text-sm lg:flex-1 lg:basis-24 lg:grow-0 lg:right-0 text-center font-semibold`}>
-            <KeywordPosition position={position} />
+            <KeywordPosition position={position} updating={updating} />
            {!updating && positionChange > 0 && <i className=' not-italic ml-1 text-xs text-[#5ed7c3]'>▲ {positionChange}</i>}
            {!updating && positionChange < 0 && <i className=' not-italic ml-1 text-xs text-red-300'>▼ {positionChange}</i>}
         </div>
--- a/components/keywords/KeywordFilter.tsx
+++ b/components/keywords/KeywordFilter.tsx
@@ -23,7 +23,7 @@ const KeywordFilters = (props: KeywordFilterProps) => {
      setDevice,
      filterKeywords,
      allTags = [],
-      keywords,
+      keywords = [],
      updateSort,
      sortBy,
      filterParams,
@@ -35,10 +35,17 @@ const KeywordFilters = (props: KeywordFilterProps) => {
   const [filterOptions, showFilterOptions] = useState(false);

   const keywordCounts = useMemo(() => {
-      return keywords.reduce((acc, k) => ({
-         desktop: k.device === 'desktop' ? acc.desktop + 1 : acc.desktop,
-         mobile: k.device !== 'desktop' ? acc.mobile + 1 : acc.mobile,
-      }), { desktop: 0, mobile: 0 });
+      const counts = { desktop: 0, mobile: 0 };
+      if (keywords && keywords.length > 0) {
+         keywords.forEach((k) => {
+            if (k.device === 'desktop') {
+               counts.desktop += 1;
+            } else {
+               counts.mobile += 1;
+            }
+         });
+      }
+      return counts;
   }, [keywords]);

   const filterCountry = (cntrs:string[]) => filterKeywords({ ...filterParams, countries: cntrs });
--- a/components/keywords/KeywordsTable.tsx
+++ b/components/keywords/KeywordsTable.tsx
@@ -1,8 +1,6 @@
 import React, { useState, useMemo } from 'react';
 import { Toaster } from 'react-hot-toast';
-import { CSSTransition } from 'react-transition-group';
 import { FixedSizeList as List, ListChildComponentProps } from 'react-window';
-import AddKeywords from './AddKeywords';
 import { filterKeywords, keywordsByDevice, sortKeywords } from '../../utils/client/sortFilter';
 import Icon from '../common/Icon';
 import Keyword from './Keyword';
@@ -25,7 +23,7 @@ type KeywordsTableProps = {
 }

 const KeywordsTable = (props: KeywordsTableProps) => {
-   const { domain, keywords = [], isLoading = true, showAddModal = false, setShowAddModal, isConsoleIntegrated = false } = props;
+   const { keywords = [], isLoading = true, isConsoleIntegrated = false } = props;
   const showSCData = isConsoleIntegrated;
   const [device, setDevice] = useState<string>('desktop');
   const [selectedKeywords, setSelectedKeywords] = useState<number[]>([]);
@@ -243,13 +241,6 @@ const KeywordsTable = (props: KeywordsTableProps) => {
                  </div>
            </Modal>
         )}
-         <CSSTransition in={showAddModal} timeout={300} classNames="modal_anim" unmountOnExit mountOnEnter>
-            <AddKeywords
-               domain={domain?.domain || ''}
-               keywords={keywords}
-               closeModal={() => setShowAddModal(false)}
-               />
-         </CSSTransition>
         {showTagManager && (
            <KeywordTagManager
               allTags={allDomainTags}
--- a/components/keywords/SCKeywordsTable.tsx
+++ b/components/keywords/SCKeywordsTable.tsx
@@ -31,7 +31,7 @@ const SCKeywordsTable = ({ domain, keywords = [], isLoading = true, isConsoleInt
   const [filterParams, setFilterParams] = useState<KeywordFilters>({ countries: [], tags: [], search: '' });
   const [sortBy, setSortBy] = useState<string>('imp_desc');
   const [SCListHeight, setSCListHeight] = useState(500);
-   const { keywordsData } = useFetchKeywords(router);
+   const { keywordsData } = useFetchKeywords(router, domain?.domain || '');
   const addedkeywords: string[] = keywordsData?.keywords?.map((key: KeywordType) => `${key.keyword}:${key.country}:${key.device}`) || [];
   const { mutate: addKeywords } = useAddKeywords(() => { if (domain && domain.slug) router.push(`/domain/${domain.slug}`); });
   const [isMobile] = useIsMobile();
@@ -209,7 +209,7 @@ const SCKeywordsTable = ({ domain, keywords = [], isLoading = true, isConsoleInt
                     )}
                     {!isConsoleIntegrated && (
                        <p className=' p-9 pt-[10%] text-center text-gray-500'>
-                        Google Search has not been Integrated yet. Please follow <a className='text-indigo-600 underline' href='https://docs.serpbear.com/miscellaneous/integrate-google-search-console' target="_blank" rel='noreferrer'>These Steps</a> to integrate Google Search Data for this Domain.
+                        Google Search Console has not been Integrated yet. Please follow <a className='text-indigo-600 underline' href='https://docs.serpbear.com/miscellaneous/integrate-google-search-console' target="_blank" rel='noreferrer'>These Steps</a> to integrate Google Search Data for this Domain.
                        </p>
                     )}
                  </div>
--- a/components/settings/NotificationSettings.tsx
+++ b/components/settings/NotificationSettings.tsx
@@ -1,6 +1,7 @@
 import React from 'react';
 import SelectField from '../common/SelectField';
 import SecretField from '../common/SecretField';
+import InputField from '../common/InputField';

 type NotificationSettingsProps = {
   settings: SettingsType,
@@ -18,8 +19,8 @@ const NotificationSettings = ({ settings, settingsError, updateSettings }:Notifi
      <div>
         <div className='settings__content styled-scrollbar p-6 text-sm'>
            <div className="settings__section__input mb-5">
-               <label className={labelStyle}>Notification Frequency</label>
               <SelectField
+               label='Notification Frequency'
                  multiple={false}
                  selected={[settings.notification_interval]}
                  options={[
@@ -32,68 +33,61 @@ const NotificationSettings = ({ settings, settingsError, updateSettings }:Notifi
                  updateField={(updated:string[]) => updated[0] && updateSettings('notification_interval', updated[0])}
                  rounded='rounded'
                  maxHeight={48}
-                  minWidth={270}
+                  minWidth={220}
               />
            </div>
            {settings.notification_interval !== 'never' && (
               <>
                  <div className="settings__section__input mb-5">
-                     <label className={labelStyle}>Notification Emails</label>
-                     <input
-                        className={`w-full p-2 border border-gray-200 rounded mb-3 focus:outline-none  focus:border-blue-200 
-                        ${settingsError?.type === 'no_email' ? ' border-red-400 focus:border-red-400' : ''} `}
-                        type="text"
-                        value={settings?.notification_email}
-                        placeholder={'test@gmail.com'}
-                        onChange={(event) => updateSettings('notification_email', event.target.value)}
+                     <InputField
+                     label='Notification Emails'
+                     hasError={settingsError?.type === 'no_email'}
+                     value={settings?.notification_email}
+                     placeholder={'test@gmail.com, test2@test.com'}
+                     onChange={(value:string) => updateSettings('notification_email', value)}
                     />
                  </div>
                  <div className="settings__section__input mb-5">
-                     <label className={labelStyle}>SMTP Server</label>
-                     <input
-                        className={`w-full p-2 border border-gray-200 rounded mb-3 focus:outline-none  focus:border-blue-200 
-                        ${settingsError?.type === 'no_smtp_server' ? ' border-red-400 focus:border-red-400' : ''} `}
-                        type="text"
-                        value={settings?.smtp_server || ''}
-                        onChange={(event) => updateSettings('smtp_server', event.target.value)}
+                     <InputField
+                     label='SMTP Server'
+                     hasError={settingsError?.type === 'no_smtp_server'}
+                     value={settings?.smtp_server || ''}
+                     placeholder={'test@gmail.com, test2@test.com'}
+                     onChange={(value:string) => updateSettings('smtp_server', value)}
                     />
                  </div>
                  <div className="settings__section__input mb-5">
-                     <label className={labelStyle}>SMTP Port</label>
-                     <input
-                        className={`w-full p-2 border border-gray-200 rounded mb-3 focus:outline-none  focus:border-blue-200 
-                        ${settingsError && settingsError.type === 'no_smtp_port' ? ' border-red-400 focus:border-red-400' : ''} `}
-                        type="text"
-                        value={settings?.smtp_port || ''}
-                        onChange={(event) => updateSettings('smtp_port', event.target.value)}
+                     <InputField
+                     label='SMTP Port'
+                     hasError={settingsError?.type === 'no_smtp_port'}
+                     value={settings?.smtp_port || ''}
+                     placeholder={'2234'}
+                     onChange={(value:string) => updateSettings('smtp_port', value)}
                     />
                  </div>
                  <div className="settings__section__input mb-5">
-                     <label className={labelStyle}>SMTP Username</label>
-                     <input
-                        className={'w-full p-2 border border-gray-200 rounded mb-3 focus:outline-none  focus:border-blue-200'}
-                        type="text"
+                     <InputField
+                        label='SMTP Username'
+                        hasError={settingsError?.type === 'no_smtp_port'}
                        value={settings?.smtp_username || ''}
-                        onChange={(event) => updateSettings('smtp_username', event.target.value)}
+                        onChange={(value:string) => updateSettings('smtp_username', value)}
+                        />
+                  </div>
+                  <div className="settings__section__input mb-5">
+                     <SecretField
+                     label='SMTP Password'
+                     value={settings?.smtp_password || ''}
+                     onChange={(value:string) => updateSettings('smtp_password', value)}
                     />
                  </div>
-
-                  <SecretField
-                  label='SMTP Password'
-                  value={settings?.smtp_password || ''}
-                  onChange={(value:string) => updateSettings('smtp_password', value)}
-                  />
-
                  <div className="settings__section__input mb-5">
-                     <label className={labelStyle}>From Email Address</label>
-                     <input
-                        className={`w-full p-2 border border-gray-200 rounded mb-3 focus:outline-none  focus:border-blue-200 
-                        ${settingsError?.type === 'no_smtp_from' ? ' border-red-400 focus:border-red-400' : ''} `}
-                        type="text"
+                        <InputField
+                        label='From Email Address'
+                        hasError={settingsError?.type === 'no_smtp_from'}
                        value={settings?.notification_email_from || ''}
                        placeholder="no-reply@mydomain.com"
-                        onChange={(event) => updateSettings('notification_email_from', event.target.value)}
-                     />
+                        onChange={(value:string) => updateSettings('notification_email_from', value)}
+                        />
                  </div>
               </>
            )}
--- a/components/settings/ScraperSettings.tsx
+++ b/components/settings/ScraperSettings.tsx
@@ -3,6 +3,7 @@ import { useClearFailedQueue } from '../../services/settings';
 import Icon from '../common/Icon';
 import SelectField, { SelectionOption } from '../common/SelectField';
 import SecretField from '../common/SecretField';
+import ToggleField from '../common/ToggleField';

 type ScraperSettingsProps = {
   settings: SettingsType,
@@ -44,18 +45,18 @@ const ScraperSettings = ({ settings, settingsError, updateSettings }:ScraperSett
      <div className='settings__content styled-scrollbar p-6 text-sm'>

         <div className="settings__section__select mb-5">
-            <label className={labelStyle}>Scraping Method</label>
            <SelectField
+            label='Scraping Method'
            options={scraperOptions}
            selected={[settings.scraper_type || 'none']}
            defaultLabel="Select Scraper"
            updateField={(updatedTime:[string]) => updateSettings('scraper_type', updatedTime[0])}
            multiple={false}
            rounded={'rounded'}
-            minWidth={270}
+            minWidth={220}
            />
         </div>
-         {['scrapingant', 'scrapingrobot', 'serply', 'serpapi', 'spaceSerp', 'searchapi'].includes(settings.scraper_type) && (
+         {settings.scraper_type !== 'none' && settings.scraper_type !== 'proxy' && (
            <SecretField
            label='Scraper API Key or Token'
            placeholder={'API Key/Token'}
@@ -79,8 +80,8 @@ const ScraperSettings = ({ settings, settingsError, updateSettings }:ScraperSett
         )}
         {settings.scraper_type !== 'none' && (
            <div className="settings__section__input mb-5">
-               <label className={labelStyle}>Scraping Frequency</label>
               <SelectField
+                  label='Scraping Frequency'
                  multiple={false}
                  selected={[settings?.scrape_interval || 'daily']}
                  options={scrapingOptions}
@@ -88,14 +89,14 @@ const ScraperSettings = ({ settings, settingsError, updateSettings }:ScraperSett
                  updateField={(updated:string[]) => updated[0] && updateSettings('scrape_interval', updated[0])}
                  rounded='rounded'
                  maxHeight={48}
-                  minWidth={270}
+                  minWidth={220}
               />
               <small className=' text-gray-500 pt-2 block'>This option requires Server/Docker Instance Restart to take Effect.</small>
            </div>
         )}
            <div className="settings__section__input mb-5">
-               <label className={labelStyle}>Delay Between Each keyword Scrape</label>
               <SelectField
+                  label='keyword Scrape Delay'
                  multiple={false}
                  selected={[settings?.scrape_delay || '0']}
                  options={delayOptions}
@@ -103,28 +104,16 @@ const ScraperSettings = ({ settings, settingsError, updateSettings }:ScraperSett
                  updateField={(updated:string[]) => updated[0] && updateSettings('scrape_delay', updated[0])}
                  rounded='rounded'
                  maxHeight={48}
-                  minWidth={270}
+                  minWidth={220}
               />
               <small className=' text-gray-500 pt-2 block'>This option requires Server/Docker Instance Restart to take Effect.</small>
            </div>
            <div className="settings__section__input mb-5">
-               <label className="relative inline-flex items-center cursor-pointer w-full justify-between">
-                  <span className="text-sm font-medium text-gray-900 dark:text-gray-300 w-56">Auto Retry Failed Keyword Scrape</span>
-                  <input
-                  type="checkbox"
-                  value={settings?.scrape_retry ? 'true' : '' }
-                  checked={settings.scrape_retry || false}
-                  className="sr-only peer"
-                  onChange={() => updateSettings('scrape_retry', !settings.scrape_retry)}
-                  />
-                  <div className="relative rounded-3xl w-9 h-5 bg-gray-200 peer-focus:outline-none peer-focus:ring-4
-                  peer-focus:ring-blue-300 dark:peer-focus:ring-blue-800rounded-full peer dark:bg-gray-700
-                  peer-checked:after:translate-x-full peer-checked:after:border-white after:content-['']
-                  after:absolute after:top-[2px] after:left-[2px] after:bg-white  after:border-gray-300
-                  after:border after:rounded-full after:h-4 after:w-4
-                  after:transition-all dark:border-gray-600 peer-checked:bg-blue-600"></div>
-
-               </label>
+               <ToggleField
+               label='Auto Retry Failed Keyword Scrape'
+               value={settings?.scrape_retry ? 'true' : '' }
+               onChange={(val) => updateSettings('scrape_retry', val)}
+               />
            </div>
            {settings?.scrape_retry && (settings.failed_queue?.length || 0) > 0 && (
               <div className="settings__section__input mb-5">
--- a/components/settings/SearchConsoleSettings.tsx
+++ b/components/settings/SearchConsoleSettings.tsx
@@ -0,0 +1,49 @@
+import React from 'react';
+import ToggleField from '../common/ToggleField';
+import InputField from '../common/InputField';
+
+type SearchConsoleSettingsProps = {
+   settings: SettingsType,
+   settingsError: null | {
+      type: string,
+      msg: string
+   },
+   updateSettings: Function,
+}
+
+const SearchConsoleSettings = ({ settings, settingsError, updateSettings }:SearchConsoleSettingsProps) => {
+   return (
+      <div>
+      <div className='settings__content styled-scrollbar p-6 text-sm'>
+
+         {/* <div className="settings__section__input mb-5">
+            <ToggleField
+            label='Enable Goolge Search Console'
+            value={settings?.scrape_retry ? 'true' : '' }
+            onChange={(val) => updateSettings('scrape_retry', val)}
+            />
+         </div> */}
+         <div className="settings__section__input mb-4 flex justify-between items-center w-full">
+            <InputField
+            label='Search Console Client Email'
+            onChange={(client_email:string) => updateSettings('search_console_client_email', client_email)}
+            value={settings.search_console_client_email}
+            placeholder='myapp@appspot.gserviceaccount.com'
+            />
+         </div>
+         <div className="settings__section__input mb-4 flex flex-col justify-between items-center w-full">
+            <label className='mb-2 font-semibold block text-sm text-gray-700 capitalize w-full'>Search Console Private Key</label>
+            <textarea
+               className={`w-full p-2 border border-gray-200 rounded mb-3 text-xs 
+               focus:outline-none h-[100px] focus:border-blue-200`}
+               value={settings.search_console_private_key}
+               placeholder={'-----BEGIN PRIVATE KEY-----/ssssaswdkihad....'}
+               onChange={(event) => updateSettings('search_console_private_key', event.target.value)}
+            />
+         </div>
+      </div>
+   </div>
+   );
+};
+
+export default SearchConsoleSettings;
--- a/components/settings/Settings.tsx
+++ b/components/settings/Settings.tsx
@@ -5,6 +5,7 @@ import Icon from '../common/Icon';
 import NotificationSettings from './NotificationSettings';
 import ScraperSettings from './ScraperSettings';
 import useOnKey from '../../hooks/useOnKey';
+import SearchConsoleSettings from './SearchConsoleSettings';

 type SettingsProps = {
   closeSettings: Function,
@@ -16,7 +17,7 @@ type SettingsError = {
   msg: string
 }

-const defaultSettings = {
+const defaultSettings: SettingsType = {
   scraper_type: 'none',
   scrape_delay: 'none',
   scrape_retry: false,
@@ -27,6 +28,9 @@ const defaultSettings = {
   smtp_username: '',
   smtp_password: '',
   notification_email_from: '',
+   search_console: true,
+   search_console_client_email: '',
+   search_console_private_key: '',
 };

 const Settings = ({ closeSettings }:SettingsProps) => {
@@ -81,30 +85,38 @@ const Settings = ({ closeSettings }:SettingsProps) => {
      }
   };

-   const tabStyle = 'inline-block px-4 py-1 rounded-full mr-3 cursor-pointer text-sm';
+   const tabStyle = `inline-block px-3 py-2 rounded-md  cursor-pointer text-xs lg:text-sm lg:mr-3 lg:px-4 select-none z-10
+   text-gray-600 border border-b-0 relative top-[1px] rounded-b-none`;
+   const tabStyleActive = 'bg-white text-blue-600 border-slate-200';
+
   return (
       <div className="settings fixed w-full h-screen top-0 left-0 z-50" onClick={closeOnBGClick}>
-            <div className="absolute w-full max-w-xs bg-white customShadow top-0 right-0 h-screen" data-loading={isLoading} >
+            <div className="absolute w-full max-w-md bg-white customShadow top-0 right-0 h-screen" data-loading={isLoading} >
               {isLoading && <div className='absolute flex content-center items-center h-full'><Icon type="loading" size={24} /></div>}
-               <div className='settings__header p-6 border-b border-b-slate-200 text-slate-500'>
+               <div className='settings__header px-5 py-4 text-slate-500'>
                  <h3 className=' text-black text-lg font-bold'>Settings</h3>
                  <button
-                  className=' absolute top-2 right-2 p-2 px-3 text-gray-400 hover:text-gray-700 transition-all hover:rotate-90'
+                  className=' absolute top-2 right-2 p-2 px- text-gray-400 hover:text-gray-700 transition-all hover:rotate-90'
                  onClick={() => closeSettings()}>
                     <Icon type='close' size={24} />
                  </button>
               </div>
-               <div className=' px-4 mt-4 '>
+               <div className='border border-slate-200 px-3 py-4 pb-0 border-l-0 border-r-0 bg-[#f8f9ff]'>
                  <ul>
                     <li
-                     className={`${tabStyle} ${currentTab === 'scraper' ? ' bg-blue-50 text-blue-600' : ''}`}
+                     className={`${tabStyle} ${currentTab === 'scraper' ? tabStyleActive : 'border-transparent '}`}
                     onClick={() => setCurrentTab('scraper')}>
-                        Scraper
+                       <Icon type='scraper' /> Scraper
                     </li>
                     <li
-                     className={`${tabStyle} ${currentTab === 'notification' ? ' bg-blue-50 text-blue-600' : ''}`}
+                     className={`${tabStyle} ${currentTab === 'notification' ? tabStyleActive : 'border-transparent'}`}
                     onClick={() => setCurrentTab('notification')}>
-                        Notification
+                        <Icon type='email' /> Notification
+                     </li>
+                     <li
+                     className={`${tabStyle} ${currentTab === 'searchconsole' ? tabStyleActive : 'border-transparent'}`}
+                     onClick={() => setCurrentTab('searchconsole')}>
+                       <Icon type='google' size={14} /> Search Console
                     </li>
                  </ul>
               </div>
@@ -115,6 +127,9 @@ const Settings = ({ closeSettings }:SettingsProps) => {
               {currentTab === 'notification' && settings && (
                  <NotificationSettings settings={settings} updateSettings={updateSettings} settingsError={settingsError} />
               )}
+               {currentTab === 'searchconsole' && settings && (
+                  <SearchConsoleSettings settings={settings} updateSettings={updateSettings} settingsError={settingsError} />
+               )}
               <div className=' border-t-[1px] border-gray-200 p-2 px-3'>
                  <button
                  onClick={() => performUpdate()}
--- a/database/config.js
+++ b/database/config.js
@@ -0,0 +1,14 @@
+module.exports = {
+  production: {
+    username: process.env.USER_NAME ? process.env.USER_NAME : process.env.USER,
+    password: process.env.PASSWORD,
+    database: 'sequelize',
+    host: 'database',
+    port: 3306,
+    dialect: 'sqlite',
+    storage: './data/database.sqlite',
+    dialectOptions: {
+      bigNumberStrings: true,
+    },
+  },
+};
--- a/database/migrations/1707068556345-add-new-keyword-fields.js
+++ b/database/migrations/1707068556345-add-new-keyword-fields.js
@@ -0,0 +1,45 @@
+// Migration: Adds city, latlong and settings keyword to keyword table.
+
+// CLI Migration
+module.exports = {
+   up: async (queryInterface, Sequelize) => {
+      return queryInterface.sequelize.transaction(async (t) => {
+         try {
+            const keywordTableDefinition = await queryInterface.describeTable('keyword');
+            if (keywordTableDefinition) {
+               if (!keywordTableDefinition.city) {
+                  await queryInterface.addColumn('keyword', 'city', { type: Sequelize.DataTypes.STRING }, { transaction: t });
+               }
+               if (!keywordTableDefinition.latlong) {
+                  await queryInterface.addColumn('keyword', 'latlong', { type: Sequelize.DataTypes.STRING }, { transaction: t });
+               }
+               if (!keywordTableDefinition.settings) {
+                  await queryInterface.addColumn('keyword', 'settings', { type: Sequelize.DataTypes.STRING }, { transaction: t });
+               }
+            }
+         } catch (error) {
+            console.log('error :', error);
+         }
+      });
+   },
+   down: (queryInterface) => {
+      return queryInterface.sequelize.transaction(async (t) => {
+         try {
+            const keywordTableDefinition = await queryInterface.describeTable('keyword');
+            if (keywordTableDefinition) {
+               if (keywordTableDefinition.city) {
+                  await queryInterface.removeColumn('keyword', 'city', { transaction: t });
+               }
+               if (keywordTableDefinition.latlong) {
+                  await queryInterface.removeColumn('keyword', 'latlong', { transaction: t });
+               }
+               if (keywordTableDefinition.latlong) {
+                  await queryInterface.removeColumn('keyword', 'settings', { transaction: t });
+               }
+            }
+         } catch (error) {
+            console.log('error :', error);
+         }
+      });
+   },
+ };
--- a/database/migrations/1707233039698-add-domain-searchconsole-field.js
+++ b/database/migrations/1707233039698-add-domain-searchconsole-field.js
@@ -0,0 +1,29 @@
+// Migration: Adds search_console field to domain table to assign search console property type, url and api.
+
+// CLI Migration
+module.exports = {
+   up: (queryInterface, Sequelize) => {
+     return queryInterface.sequelize.transaction(async (t) => {
+      try {
+         const domainTableDefinition = await queryInterface.describeTable('domain');
+         if (domainTableDefinition && !domainTableDefinition.search_console) {
+            await queryInterface.addColumn('domain', 'search_console', { type: Sequelize.DataTypes.STRING }, { transaction: t });
+         }
+      } catch (error) {
+         console.log('error :', error);
+      }
+     });
+   },
+   down: (queryInterface) => {
+      return queryInterface.sequelize.transaction(async (t) => {
+         try {
+            const domainTableDefinition = await queryInterface.describeTable('domain');
+            if (domainTableDefinition && domainTableDefinition.search_console) {
+               await queryInterface.removeColumn('domain', 'search_console', { transaction: t });
+            }
+         } catch (error) {
+            console.log('error :', error);
+         }
+      });
+   },
+ };
--- a/database/models/domain.ts
+++ b/database/models/domain.ts
@@ -38,6 +38,9 @@ class Domain extends Model {

   @Column({ type: DataType.STRING, allowNull: true, defaultValue: '' })
   notification_emails!: string;
+
+   @Column({ type: DataType.STRING, allowNull: true })
+   search_console!: string;
 }

 export default Domain;
--- a/database/models/keyword.ts
+++ b/database/models/keyword.ts
@@ -19,7 +19,13 @@ class Keyword extends Model {
   @Column({ type: DataType.STRING, allowNull: true, defaultValue: 'US' })
   country!: string;

-   @Column({ type: DataType.STRING, allowNull: false })
+   @Column({ type: DataType.STRING, allowNull: true, defaultValue: '' })
+   city!: string;
+
+   @Column({ type: DataType.STRING, allowNull: true, defaultValue: '' })
+   latlong!: string;
+
+   @Column({ type: DataType.STRING, allowNull: false, defaultValue: '{}' })
   domain!: string;

   // @ForeignKey(() => Domain)
@@ -58,6 +64,9 @@ class Keyword extends Model {

   @Column({ type: DataType.STRING, allowNull: true, defaultValue: 'false' })
   lastUpdateError!: string;
+
+   @Column({ type: DataType.STRING, allowNull: true })
+   settings!: string;
 }

 export default Keyword;
--- a/entrypoint.sh
+++ b/entrypoint.sh
@@ -0,0 +1,3 @@
+#!/bin/sh
+npx sequelize-cli db:migrate --env production
+exec "$@"
--- a/package-lock.json
+++ b/package-lock.json
--- a/package.json
+++ b/package.json
@@ -1,6 +1,6 @@
 {
  "name": "serpbear",
-  "version": "0.3.4",
+  "version": "1.0.1",
  "private": true,
  "scripts": {
    "dev": "next dev",
@@ -13,6 +13,8 @@
    "test": "jest --watch --verbose",
    "test:ci": "jest --ci",
    "test:cv": "jest --coverage --coverageDirectory='coverage'",
+    "db:migrate": "sequelize-cli db:migrate --env production",
+    "db:revert": "sequelize-cli db:migrate:undo --env production",
    "release": "standard-version"
  },
  "dependencies": {
@@ -33,7 +35,7 @@
    "jsonwebtoken": "^9.0.2",
    "msw": "^0.49.0",
    "next": "^12.3.4",
-    "nodemailer": "^6.8.0",
+    "nodemailer": "^6.9.9",
    "react": "18.2.0",
    "react-chartjs-2": "^4.3.1",
    "react-dom": "18.2.0",
@@ -44,8 +46,9 @@
    "react-window": "^1.8.8",
    "reflect-metadata": "^0.1.13",
    "sequelize": "^6.34.0",
-    "sequelize-typescript": "^2.1.5",
-    "sqlite3": "^5.1.6"
+    "sequelize-typescript": "^2.1.6",
+    "sqlite3": "^5.1.6",
+    "umzug": "^3.6.1"
  },
  "devDependencies": {
    "@testing-library/jest-dom": "^6.1.4",
@@ -73,6 +76,7 @@
    "prettier": "^2.7.1",
    "resize-observer-polyfill": "^1.5.1",
    "sass": "^1.55.0",
+    "sequelize-cli": "^6.6.2",
    "standard-version": "^9.5.0",
    "stylelint-config-standard": "^29.0.0",
    "tailwindcss": "^3.1.8",
--- a/pages/api/dbmigrate.ts
+++ b/pages/api/dbmigrate.ts
@@ -0,0 +1,53 @@
+import { Sequelize } from 'sequelize';
+import { Umzug, SequelizeStorage } from 'umzug';
+import type { NextApiRequest, NextApiResponse } from 'next';
+import db from '../../database/database';
+import verifyUser from '../../utils/verifyUser';
+
+type MigrationGetResponse = {
+   hasMigrations: boolean,
+}
+
+type MigrationPostResponse = {
+   migrated: boolean,
+   erroor?: string
+}
+
+export default async function handler(req: NextApiRequest, res: NextApiResponse) {
+   const authorized = verifyUser(req, res);
+   if (authorized === 'authorized' && req.method === 'GET') {
+      await db.sync();
+      return getMigrationStatus(req, res);
+   }
+   if (authorized === 'authorized' && req.method === 'POST') {
+      return migrateDatabase(req, res);
+   }
+   return res.status(401).json({ error: authorized });
+}
+
+const getMigrationStatus = async (req: NextApiRequest, res: NextApiResponse<MigrationGetResponse>) => {
+   const sequelize = new Sequelize({ dialect: 'sqlite', storage: './data/database.sqlite', logging: false });
+   const umzug = new Umzug({
+      migrations: { glob: 'database/migrations/*.js' },
+      context: sequelize.getQueryInterface(),
+      storage: new SequelizeStorage({ sequelize }),
+      logger: undefined,
+   });
+   const migrations = await umzug.pending();
+   // console.log('migrations :', migrations);
+   // const migrationsExceuted = await umzug.executed();
+   return res.status(200).json({ hasMigrations: migrations.length > 0 });
+};
+
+const migrateDatabase = async (req: NextApiRequest, res: NextApiResponse<MigrationPostResponse>) => {
+   const sequelize = new Sequelize({ dialect: 'sqlite', storage: './data/database.sqlite', logging: false });
+   const umzug = new Umzug({
+      migrations: { glob: 'database/migrations/*.js' },
+      context: sequelize.getQueryInterface(),
+      storage: new SequelizeStorage({ sequelize }),
+      logger: undefined,
+   });
+   const migrations = await umzug.up();
+   console.log('[Updated] migrations :', migrations);
+   return res.status(200).json({ migrated: true });
+};
--- a/pages/api/domain.ts
+++ b/pages/api/domain.ts
@@ -0,0 +1,48 @@
+import type { NextApiRequest, NextApiResponse } from 'next';
+import Cryptr from 'cryptr';
+import db from '../../database/database';
+import Domain from '../../database/models/domain';
+import verifyUser from '../../utils/verifyUser';
+
+type DomainGetResponse = {
+   domain?: DomainType | null
+   error?: string|null,
+}
+
+export default async function handler(req: NextApiRequest, res: NextApiResponse) {
+   const authorized = verifyUser(req, res);
+   if (authorized === 'authorized' && req.method === 'GET') {
+      await db.sync();
+      return getDomain(req, res);
+   }
+   return res.status(401).json({ error: authorized });
+}
+
+const getDomain = async (req: NextApiRequest, res: NextApiResponse<DomainGetResponse>) => {
+   if (!req.query.domain && typeof req.query.domain !== 'string') {
+       return res.status(400).json({ error: 'Domain Name is Required!' });
+   }
+
+   try {
+      const query = { domain: req.query.domain as string };
+      const foundDomain:Domain| null = await Domain.findOne({ where: query });
+      const parsedDomain = foundDomain?.get({ plain: true }) || false;
+
+      if (parsedDomain && parsedDomain.search_console) {
+         try {
+            const cryptr = new Cryptr(process.env.SECRET as string);
+            const scData = JSON.parse(parsedDomain.search_console);
+            scData.client_email = scData.client_email ? cryptr.decrypt(scData.client_email) : '';
+            scData.private_key = scData.private_key ? cryptr.decrypt(scData.private_key) : '';
+            parsedDomain.search_console = JSON.stringify(scData);
+         } catch (error) {
+            console.log('[Error] Parsing Search Console Keys.');
+         }
+      }
+
+      return res.status(200).json({ domain: parsedDomain });
+   } catch (error) {
+      console.log('[ERROR] Getting Domain: ', error);
+      return res.status(400).json({ error: 'Error Loading Domain' });
+   }
+};
--- a/pages/api/domains.ts
+++ b/pages/api/domains.ts
@@ -1,10 +1,11 @@
 import type { NextApiRequest, NextApiResponse } from 'next';
+import Cryptr from 'cryptr';
 import db from '../../database/database';
 import Domain from '../../database/models/domain';
 import Keyword from '../../database/models/keyword';
 import getdomainStats from '../../utils/domains';
 import verifyUser from '../../utils/verifyUser';
-import { removeLocalSCData } from '../../utils/searchConsole';
+import { checkSerchConsoleIntegration, removeLocalSCData } from '../../utils/searchConsole';

 type DomainsGetRes = {
   domains: DomainType[]
@@ -53,7 +54,13 @@ export const getDomains = async (req: NextApiRequest, res: NextApiResponse<Domai
   const withStats = !!req?.query?.withstats;
   try {
      const allDomains: Domain[] = await Domain.findAll();
-      const formattedDomains: DomainType[] = allDomains.map((el) => el.get({ plain: true }));
+      const formattedDomains: DomainType[] = allDomains.map((el) => {
+         const domainItem:DomainType = el.get({ plain: true });
+         const scData = domainItem?.search_console ? JSON.parse(domainItem.search_console) : {};
+         const { client_email, private_key } = scData;
+         const searchConsoleData = scData ? { ...scData, client_email: client_email ? 'true' : '', private_key: private_key ? 'true' : '' } : {};
+         return { ...domainItem, search_console: JSON.stringify(searchConsoleData) };
+      });
      const theDomains: DomainType[] = withStats ? await getdomainStats(formattedDomains) : allDomains;
      return res.status(200).json({ domains: theDomains });
   } catch (error) {
@@ -69,7 +76,7 @@ const addDomain = async (req: NextApiRequest, res: NextApiResponse<DomainsAddRes
      domains.forEach((domain: string) => {
         domainsToAdd.push({
            domain: domain.trim(),
-            slug: domain.trim().replaceAll('-', '_').replaceAll('.', '-'),
+            slug: domain.trim().replaceAll('-', '_').replaceAll('.', '-').replaceAll('/', '-'),
            lastUpdated: new Date().toJSON(),
            added: new Date().toJSON(),
         });
@@ -108,17 +115,28 @@ export const updateDomain = async (req: NextApiRequest, res: NextApiResponse<Dom
      return res.status(400).json({ domain: null, error: 'Domain is Required!' });
   }
   const { domain } = req.query || {};
-   const { notification_interval, notification_emails } = req.body;
+   const { notification_interval, notification_emails, search_console } = req.body as DomainSettings;

   try {
      const domainToUpdate: Domain|null = await Domain.findOne({ where: { domain } });
+      // Validate Search Console API Data
+      if (domainToUpdate && search_console?.client_email && search_console?.private_key) {
+         const theDomainObj = domainToUpdate.get({ plain: true });
+         const isSearchConsoleAPIValid = await checkSerchConsoleIntegration({ ...theDomainObj, search_console: JSON.stringify(search_console) });
+         if (!isSearchConsoleAPIValid.isValid) {
+            return res.status(400).json({ domain: null, error: isSearchConsoleAPIValid.error });
+         }
+         const cryptr = new Cryptr(process.env.SECRET as string);
+         search_console.client_email = search_console.client_email ? cryptr.encrypt(search_console.client_email.trim()) : '';
+         search_console.private_key = search_console.private_key ? cryptr.encrypt(search_console.private_key.trim()) : '';
+      }
      if (domainToUpdate) {
-         domainToUpdate.set({ notification_interval, notification_emails });
+         domainToUpdate.set({ notification_interval, notification_emails, search_console: JSON.stringify(search_console) });
         await domainToUpdate.save();
      }
      return res.status(200).json({ domain: domainToUpdate });
   } catch (error) {
      console.log('[ERROR] Updating Domain: ', req.query.domain, error);
-      return res.status(400).json({ domain: null, error: 'Error Updating Domain' });
+      return res.status(400).json({ domain: null, error: 'Error Updating Domain. An Unknown Error Occured.' });
   }
 };
--- a/pages/api/insight.ts
+++ b/pages/api/insight.ts
@@ -1,8 +1,9 @@
 import type { NextApiRequest, NextApiResponse } from 'next';
 import db from '../../database/database';
 import { getCountryInsight, getKeywordsInsight, getPagesInsight } from '../../utils/insight';
-import { fetchDomainSCData, readLocalSCData } from '../../utils/searchConsole';
+import { fetchDomainSCData, getSearchConsoleApiInfo, readLocalSCData } from '../../utils/searchConsole';
 import verifyUser from '../../utils/verifyUser';
+import Domain from '../../database/models/domain';

 type SCInsightRes = {
   data: InsightDataType | null,
@@ -23,9 +24,6 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse)

 const getDomainSearchConsoleInsight = async (req: NextApiRequest, res: NextApiResponse<SCInsightRes>) => {
   if (!req.query.domain && typeof req.query.domain !== 'string') return res.status(400).json({ data: null, error: 'Domain is Missing.' });
-   if (!!(process.env.SEARCH_CONSOLE_PRIVATE_KEY && process.env.SEARCH_CONSOLE_CLIENT_EMAIL) === false) {
-      return res.status(200).json({ data: null, error: 'Google Search Console Not Integrated' });
-   }
   const domainname = (req.query.domain as string).replaceAll('-', '.').replaceAll('_', '-');
   const getInsightFromSCData = (localSCData: SCDomainDataType): InsightDataType => {
      const { stats = [] } = localSCData;
@@ -37,17 +35,26 @@ const getDomainSearchConsoleInsight = async (req: NextApiRequest, res: NextApiRe

   // First try and read the  Local SC Domain Data file.
   const localSCData = await readLocalSCData(domainname);
-   const oldFetchedDate = localSCData.lastFetched;
-   const fetchTimeDiff = new Date().getTime() - (oldFetchedDate ? new Date(oldFetchedDate as string).getTime() : 0);

-   if (localSCData && localSCData.stats && localSCData.stats.length && fetchTimeDiff <= 86400000) {
-      const response = getInsightFromSCData(localSCData);
-      return res.status(200).json({ data: response });
+   if (localSCData) {
+      const oldFetchedDate = localSCData.lastFetched;
+      const fetchTimeDiff = new Date().getTime() - (oldFetchedDate ? new Date(oldFetchedDate as string).getTime() : 0);
+      if (localSCData.stats && localSCData.stats.length && fetchTimeDiff <= 86400000) {
+         const response = getInsightFromSCData(localSCData);
+         return res.status(200).json({ data: response });
+      }
   }

   // If the Local SC Domain Data file does not exist, fetch from Googel Search Console.
   try {
-      const scData = await fetchDomainSCData(domainname);
+      const query = { domain: domainname };
+      const foundDomain:Domain| null = await Domain.findOne({ where: query });
+      const domainObj: DomainType = foundDomain && foundDomain.get({ plain: true });
+      const scDomainAPI = await getSearchConsoleApiInfo(domainObj);
+      if (!(scDomainAPI.client_email && scDomainAPI.private_key)) {
+         return res.status(200).json({ data: null, error: 'Google Search Console is not Integrated.' });
+      }
+      const scData = await fetchDomainSCData(domainObj, scDomainAPI);
      const response = getInsightFromSCData(scData);
      return res.status(200).json({ data: response });
   } catch (error) {
--- a/pages/api/keywords.ts
+++ b/pages/api/keywords.ts
@@ -45,7 +45,7 @@ const getKeywords = async (req: NextApiRequest, res: NextApiResponse<KeywordsGet
   if (!req.query.domain && typeof req.query.domain !== 'string') {
      return res.status(400).json({ error: 'Domain is Required!' });
   }
-   const domain = (req.query.domain as string).replaceAll('-', '.').replaceAll('_', '-');
+   const domain = (req.query.domain as string);
   const integratedSC = process.env.SEARCH_CONSOLE_PRIVATE_KEY && process.env.SEARCH_CONSOLE_CLIENT_EMAIL;
   const domainSCData = integratedSC ? await readLocalSCData(domain) : false;

@@ -79,13 +79,14 @@ const addKeywords = async (req: NextApiRequest, res: NextApiResponse<KeywordsGet
      const keywordsToAdd: any = []; // QuickFIX for bug: https://github.com/sequelize/sequelize-typescript/issues/936

      keywords.forEach((kwrd: KeywordAddPayload) => {
-         const { keyword, device, country, domain, tags } = kwrd;
+         const { keyword, device, country, domain, tags, city } = kwrd;
         const tagsArray = tags ? tags.split(',').map((item:string) => item.trim()) : [];
         const newKeyword = {
            keyword,
            device,
            domain,
            country,
+            city,
            position: 0,
            updating: true,
            history: JSON.stringify({}),
--- a/pages/api/searchconsole.ts
+++ b/pages/api/searchconsole.ts
@@ -1,7 +1,7 @@
 import type { NextApiRequest, NextApiResponse } from 'next';
 import db from '../../database/database';
 import Domain from '../../database/models/domain';
-import { fetchDomainSCData, readLocalSCData } from '../../utils/searchConsole';
+import { fetchDomainSCData, getSearchConsoleApiInfo, readLocalSCData } from '../../utils/searchConsole';
 import verifyUser from '../../utils/verifyUser';

 type searchConsoleRes = {
@@ -31,18 +31,20 @@ export default async function handler(req: NextApiRequest, res: NextApiResponse)

 const getDomainSearchConsoleData = async (req: NextApiRequest, res: NextApiResponse<searchConsoleRes>) => {
   if (!req.query.domain && typeof req.query.domain !== 'string') return res.status(400).json({ data: null, error: 'Domain is Missing.' });
-   if (!!(process.env.SEARCH_CONSOLE_PRIVATE_KEY && process.env.SEARCH_CONSOLE_CLIENT_EMAIL) === false) {
-      return res.status(200).json({ data: null, error: 'Google Search Console Not Integrated' });
-   }
   const domainname = (req.query.domain as string).replaceAll('-', '.').replaceAll('_', '-');
   const localSCData = await readLocalSCData(domainname);
-   console.log(localSCData && localSCData.thirtyDays && localSCData.thirtyDays.length);
-
   if (localSCData && localSCData.thirtyDays && localSCData.thirtyDays.length) {
      return res.status(200).json({ data: localSCData });
   }
   try {
-      const scData = await fetchDomainSCData(domainname);
+      const query = { domain: domainname };
+      const foundDomain:Domain| null = await Domain.findOne({ where: query });
+      const domainObj: DomainType = foundDomain && foundDomain.get({ plain: true });
+      const scDomainAPI = await getSearchConsoleApiInfo(domainObj);
+      if (!(scDomainAPI.client_email && scDomainAPI.private_key)) {
+         return res.status(200).json({ data: null, error: 'Google Search Console is not Integrated.' });
+      }
+      const scData = await fetchDomainSCData(domainObj, scDomainAPI);
      return res.status(200).json({ data: scData });
   } catch (error) {
      console.log('[ERROR] Getting Search Console Data for: ', domainname, error);
@@ -53,9 +55,9 @@ const getDomainSearchConsoleData = async (req: NextApiRequest, res: NextApiRespo
 const cronRefreshSearchConsoleData = async (req: NextApiRequest, res: NextApiResponse<searchConsoleCRONRes>) => {
   try {
      const allDomainsRaw = await Domain.findAll();
-      const Domains: Domain[] = allDomainsRaw.map((el) => el.get({ plain: true }));
+      const Domains: DomainType[] = allDomainsRaw.map((el) => el.get({ plain: true }));
      for (const domain of Domains) {
-         await fetchDomainSCData(domain.domain);
+         await fetchDomainSCData(domain);
      }
      return res.status(200).json({ status: 'completed' });
   } catch (error) {
--- a/pages/api/settings.ts
+++ b/pages/api/settings.ts
@@ -42,9 +42,11 @@ const updateSettings = async (req: NextApiRequest, res: NextApiResponse<Settings
   }
   try {
      const cryptr = new Cryptr(process.env.SECRET as string);
-      const scaping_api = settings.scaping_api ? cryptr.encrypt(settings.scaping_api) : '';
-      const smtp_password = settings.smtp_password ? cryptr.encrypt(settings.smtp_password) : '';
-      const securedSettings = { ...settings, scaping_api, smtp_password };
+      const scaping_api = settings.scaping_api ? cryptr.encrypt(settings.scaping_api.trim()) : '';
+      const smtp_password = settings.smtp_password ? cryptr.encrypt(settings.smtp_password.trim()) : '';
+      const search_console_client_email = settings.search_console_client_email ? cryptr.encrypt(settings.search_console_client_email.trim()) : '';
+      const search_console_private_key = settings.search_console_private_key ? cryptr.encrypt(settings.search_console_private_key.trim()) : '';
+      const securedSettings = { ...settings, scaping_api, smtp_password, search_console_client_email, search_console_private_key };

      await writeFile(`${process.cwd()}/data/settings.json`, JSON.stringify(securedSettings), { encoding: 'utf-8' });
      return res.status(200).json({ settings });
@@ -67,12 +69,17 @@ export const getAppSettings = async () : Promise<SettingsType> => {
         const cryptr = new Cryptr(process.env.SECRET as string);
         const scaping_api = settings.scaping_api ? cryptr.decrypt(settings.scaping_api) : '';
         const smtp_password = settings.smtp_password ? cryptr.decrypt(settings.smtp_password) : '';
+         const search_console_client_email = settings.search_console_client_email ? cryptr.decrypt(settings.search_console_client_email) : '';
+         const search_console_private_key = settings.search_console_private_key ? cryptr.decrypt(settings.search_console_private_key) : '';
         decryptedSettings = {
            ...settings,
            scaping_api,
            smtp_password,
-            search_console_integrated: !!(process.env.SEARCH_CONSOLE_PRIVATE_KEY && process.env.SEARCH_CONSOLE_CLIENT_EMAIL),
-            available_scapers: allScrapers.map((scraper) => ({ label: scraper.name, value: scraper.id })),
+            search_console_client_email,
+            search_console_private_key,
+            search_console_integrated: !!(process.env.SEARCH_CONSOLE_PRIVATE_KEY && process.env.SEARCH_CONSOLE_CLIENT_EMAIL)
+            || !!(search_console_client_email && search_console_private_key),
+            available_scapers: allScrapers.map((scraper) => ({ label: scraper.name, value: scraper.id, allowsCity: !!scraper.allowsCity })),
            failed_queue: failedQueue,
            screenshot_key: screenshotAPIKey,
         };
@@ -94,6 +101,9 @@ export const getAppSettings = async () : Promise<SettingsType> => {
         smtp_password: '',
         scrape_retry: false,
         screenshot_key: screenshotAPIKey,
+         search_console: true,
+         search_console_client_email: '',
+         search_console_private_key: '',
      };
      const otherSettings = {
         available_scapers: allScrapers.map((scraper) => ({ label: scraper.name, value: scraper.id })),
--- a/pages/domain/[slug]/index.tsx
+++ b/pages/domain/[slug]/index.tsx
@@ -1,9 +1,7 @@
-import React, { useEffect, useMemo, useState } from 'react';
+import React, { useMemo, useState } from 'react';
 import type { NextPage } from 'next';
 import Head from 'next/head';
 import { useRouter } from 'next/router';
-// import { useQuery } from 'react-query';
-// import toast from 'react-hot-toast';
 import { CSSTransition } from 'react-transition-group';
 import Sidebar from '../../../components/common/Sidebar';
 import TopBar from '../../../components/common/TopBar';
@@ -16,21 +14,20 @@ import Settings from '../../../components/settings/Settings';
 import { useFetchDomains } from '../../../services/domains';
 import { useFetchKeywords } from '../../../services/keywords';
 import { useFetchSettings } from '../../../services/settings';
+import AddKeywords from '../../../components/keywords/AddKeywords';

 const SingleDomain: NextPage = () => {
   const router = useRouter();
-   const [noScrapprtError, setNoScrapprtError] = useState(false);
   const [showAddKeywords, setShowAddKeywords] = useState(false);
   const [showAddDomain, setShowAddDomain] = useState(false);
   const [showDomainSettings, setShowDomainSettings] = useState(false);
   const [showSettings, setShowSettings] = useState(false);
   const [keywordSPollInterval, setKeywordSPollInterval] = useState<undefined|number>(undefined);
-   const { data: appSettings } = useFetchSettings();
+   const { data: appSettingsData, isLoading: isAppSettingsLoading } = useFetchSettings();
   const { data: domainsData } = useFetchDomains(router);
-   const { keywordsData, keywordsLoading } = useFetchKeywords(router, setKeywordSPollInterval, keywordSPollInterval);
-
-   const theDomains: DomainType[] = (domainsData && domainsData.domains) || [];
-   const theKeywords: KeywordType[] = keywordsData && keywordsData.keywords;
+   const appSettings: SettingsType = appSettingsData?.settings || {};
+   const { scraper_type = '', available_scapers = [] } = appSettings;
+   const activeScraper = useMemo(() => available_scapers.find((scraper) => scraper.value === scraper_type), [scraper_type, available_scapers]);

   const activDomain: DomainType|null = useMemo(() => {
      let active:DomainType|null = null;
@@ -40,18 +37,18 @@ const SingleDomain: NextPage = () => {
      return active;
   }, [router.query.slug, domainsData]);

-   useEffect(() => {
-      // console.log('appSettings.settings: ', appSettings && appSettings.settings);
-      if (appSettings && appSettings.settings && (!appSettings.settings.scraper_type || (appSettings.settings.scraper_type === 'none'))) {
-         setNoScrapprtError(true);
-      }
-   }, [appSettings]);
+   const domainHasScAPI = useMemo(() => {
+      const doaminSc = activDomain?.search_console ? JSON.parse(activDomain.search_console) : {};
+      return !!(doaminSc?.client_email && doaminSc?.private_key);
+   }, [activDomain]);

-   // console.log('Domains Data:', router, activDomain, theKeywords);
+   const { keywordsData, keywordsLoading } = useFetchKeywords(router, activDomain?.domain || '', setKeywordSPollInterval, keywordSPollInterval);
+   const theDomains: DomainType[] = (domainsData && domainsData.domains) || [];
+   const theKeywords: KeywordType[] = keywordsData && keywordsData.keywords;

   return (
      <div className="Domain ">
-         {noScrapprtError && (
+         {((!scraper_type || (scraper_type === 'none')) && !isAppSettingsLoading) && (
               <div className=' p-3 bg-red-600 text-white text-sm text-center'>
                  A Scrapper/Proxy has not been set up Yet. Open Settings to set it up and start using the app.
               </div>
@@ -80,7 +77,7 @@ const SingleDomain: NextPage = () => {
               keywords={theKeywords}
               showAddModal={showAddKeywords}
               setShowAddModal={setShowAddKeywords}
-               isConsoleIntegrated={!!(appSettings && appSettings?.settings?.search_console_integrated) }
+               isConsoleIntegrated={!!(appSettings && appSettings.search_console_integrated) || domainHasScAPI }
               />
            </div>
         </div>
@@ -98,6 +95,15 @@ const SingleDomain: NextPage = () => {
         <CSSTransition in={showSettings} timeout={300} classNames="settings_anim" unmountOnExit mountOnEnter>
             <Settings closeSettings={() => setShowSettings(false)} />
         </CSSTransition>
+         <CSSTransition in={showAddKeywords} timeout={300} classNames="modal_anim" unmountOnExit mountOnEnter>
+            <AddKeywords
+               domain={activDomain?.domain || ''}
+               scraperName={activeScraper?.label || ''}
+               keywords={theKeywords}
+               allowsCity={!!activeScraper?.allowsCity}
+               closeModal={() => setShowAddKeywords(false)}
+               />
+         </CSSTransition>
      </div>
   );
 };
--- a/pages/domain/console/[slug]/index.tsx
+++ b/pages/domain/console/[slug]/index.tsx
@@ -39,6 +39,11 @@ const DiscoverPage: NextPage = () => {
      return active;
   }, [router.query.slug, domainsData]);

+   const domainHasScAPI = useMemo(() => {
+      const doaminSc = activDomain?.search_console ? JSON.parse(activDomain.search_console) : {};
+      return !!(doaminSc?.client_email && doaminSc?.private_key);
+   }, [activDomain]);
+
   return (
      <div className="Domain ">
         {activDomain && activDomain.domain
@@ -65,7 +70,7 @@ const DiscoverPage: NextPage = () => {
               isLoading={keywordsLoading || isFetching}
               domain={activDomain}
               keywords={theKeywords}
-               isConsoleIntegrated={scConnected}
+               isConsoleIntegrated={scConnected || domainHasScAPI}
               />
            </div>
         </div>
--- a/pages/domain/insight/[slug]/index.tsx
+++ b/pages/domain/insight/[slug]/index.tsx
@@ -39,6 +39,11 @@ const InsightPage: NextPage = () => {
      return active;
   }, [router.query.slug, domainsData]);

+   const domainHasScAPI = useMemo(() => {
+      const doaminSc = activDomain?.search_console ? JSON.parse(activDomain.search_console) : {};
+      return !!(doaminSc?.client_email && doaminSc?.private_key);
+   }, [activDomain]);
+
   return (
      <div className="Domain ">
         {activDomain && activDomain.domain
@@ -65,7 +70,7 @@ const InsightPage: NextPage = () => {
               isLoading={false}
               domain={activDomain}
               insight={theInsight}
-               isConsoleIntegrated={scConnected}
+               isConsoleIntegrated={scConnected || domainHasScAPI}
               />
            </div>
         </div>
--- a/pages/domains/index.tsx
+++ b/pages/domains/index.tsx
@@ -7,7 +7,7 @@ import toast, { Toaster } from 'react-hot-toast';
 import TopBar from '../../components/common/TopBar';
 import AddDomain from '../../components/domains/AddDomain';
 import Settings from '../../components/settings/Settings';
-import { useFetchSettings } from '../../services/settings';
+import { useCheckMigrationStatus, useFetchSettings } from '../../services/settings';
 import { fetchDomainScreenshot, useFetchDomains } from '../../services/domains';
 import DomainItem from '../../components/domains/DomainItem';
 import Icon from '../../components/common/Icon';
@@ -16,12 +16,17 @@ type thumbImages = { [domain:string] : string }

 const Domains: NextPage = () => {
   const router = useRouter();
-   const [noScrapprtError, setNoScrapprtError] = useState(false);
+   // const [noScrapprtError, setNoScrapprtError] = useState(false);
   const [showSettings, setShowSettings] = useState(false);
   const [showAddDomain, setShowAddDomain] = useState(false);
   const [domainThumbs, setDomainThumbs] = useState<thumbImages>({});
-   const { data: appSettings } = useFetchSettings();
+   const { data: appSettingsData, isLoading: isAppSettingsLoading } = useFetchSettings();
   const { data: domainsData, isLoading } = useFetchDomains(router, true);
+   const { data: migrationStatus } = useCheckMigrationStatus();
+   // const { mutate: updateDatabaseMutate, isLoading: isUpdatingDB } = useMigrateDatabase((res:Object) => { window.location.reload(); });
+
+   const appSettings:SettingsType = appSettingsData?.settings || {};
+   const { scraper_type = '' } = appSettings;

   const totalKeywords = useMemo(() => {
      let keywords = 0;
@@ -33,29 +38,33 @@ const Domains: NextPage = () => {
      return keywords;
   }, [domainsData]);

+   const domainSCAPiObj = useMemo(() => {
+      const domainsSCAPI:{ [ID:string] : boolean } = {};
+      if (domainsData?.domains) {
+         domainsData.domains.forEach(async (domain:DomainType) => {
+            const doaminSc = domain?.search_console ? JSON.parse(domain.search_console) : {};
+            domainsSCAPI[domain.ID] = doaminSc.client_email && doaminSc.private_key;
+         });
+      }
+      return domainsSCAPI;
+   }, [domainsData]);
+
   useEffect(() => {
-      if (domainsData?.domains && domainsData.domains.length > 0 && appSettings?.settings?.screenshot_key) {
+      if (domainsData?.domains && domainsData.domains.length > 0 && appSettings.screenshot_key) {
         domainsData.domains.forEach(async (domain:DomainType) => {
            if (domain.domain) {
-               const domainThumb = await fetchDomainScreenshot(domain.domain, appSettings.settings.screenshot_key);
+               const domainThumb = await fetchDomainScreenshot(domain.domain, appSettings.screenshot_key || '');
               if (domainThumb) {
                  setDomainThumbs((currentThumbs) => ({ ...currentThumbs, [domain.domain]: domainThumb }));
               }
            }
         });
      }
-   }, [domainsData, appSettings]);
-
-   useEffect(() => {
-      // console.log('appSettings.settings: ', appSettings && appSettings.settings);
-      if (appSettings && appSettings.settings && (!appSettings.settings.scraper_type || (appSettings.settings.scraper_type === 'none'))) {
-         setNoScrapprtError(true);
-      }
-   }, [appSettings]);
+   }, [domainsData, appSettings.screenshot_key]);

   const manuallyUpdateThumb = async (domain: string) => {
-      if (domain && appSettings?.settings?.screenshot_key) {
-         const domainThumb = await fetchDomainScreenshot(domain, appSettings.settings.screenshot_key, true);
+      if (domain && appSettings.screenshot_key) {
+         const domainThumb = await fetchDomainScreenshot(domain, appSettings.screenshot_key, true);
         if (domainThumb) {
            toast(`${domain} Screenshot Updated Successfully!`, { icon: '✔️' });
            setDomainThumbs((currentThumbs) => ({ ...currentThumbs, [domain]: domainThumb }));
@@ -67,11 +76,17 @@ const Domains: NextPage = () => {

   return (
      <div data-testid="domains" className="Domain flex flex-col min-h-screen">
-         {noScrapprtError && (
+         {((!scraper_type || (scraper_type === 'none')) && !isAppSettingsLoading) && (
               <div className=' p-3 bg-red-600 text-white text-sm text-center'>
                  A Scrapper/Proxy has not been set up Yet. Open Settings to set it up and start using the app.
               </div>
         )}
+         {migrationStatus?.hasMigrations && (
+               <div className=' p-3 bg-black text-white text-sm text-center'>
+                  You need to Update your database. Stop Serpbear and run this command to update your database:
+                  <code className=' bg-gray-700 px-2 py-0 ml-1'>npm run db:migrate</code>
+               </div>
+         )}
         <Head>
            <title>Domains - SerpBear</title>
         </Head>
@@ -99,7 +114,7 @@ const Domains: NextPage = () => {
                           key={domain.ID}
                           domain={domain}
                           selected={false}
-                           isConsoleIntegrated={!!(appSettings && appSettings?.settings?.search_console_integrated) }
+                           isConsoleIntegrated={!!(appSettings && appSettings.search_console_integrated) || !!domainSCAPiObj[domain.ID] }
                           thumb={domainThumbs[domain.domain]}
                           updateThumb={manuallyUpdateThumb}
                           // isConsoleIntegrated={false}
@@ -125,7 +140,7 @@ const Domains: NextPage = () => {
             <Settings closeSettings={() => setShowSettings(false)} />
         </CSSTransition>
         <footer className='text-center flex flex-1 justify-center pb-5 items-end'>
-            <span className='text-gray-500 text-xs'><a href='https://github.com/towfiqi/serpbear' target="_blank" rel='noreferrer'>SerpBear v{appSettings?.settings?.version || '0.0.0'}</a></span>
+            <span className='text-gray-500 text-xs'><a href='https://github.com/towfiqi/serpbear' target="_blank" rel='noreferrer'>SerpBear v{appSettings.version || '0.0.0'}</a></span>
         </footer>
         <Toaster position='bottom-center' containerClassName="react_toaster" />
      </div>
--- a/scrapers/index.ts
+++ b/scrapers/index.ts
@@ -5,6 +5,8 @@ import serply from './services/serply';
 import spaceserp from './services/spaceserp';
 import proxy from './services/proxy';
 import searchapi from './services/searchapi';
+import valueSerp from './services/valueserp';
+import serper from './services/serper';

 export default [
   scrapingRobot,
@@ -14,4 +16,6 @@ export default [
   spaceserp,
   proxy,
   searchapi,
+   valueSerp,
+   serper,
 ];
--- a/scrapers/services/searchapi.ts
+++ b/scrapers/services/searchapi.ts
@@ -1,7 +1,16 @@
+import countries from '../../utils/countries';
+
+interface SearchApiResult {
+   title: string,
+   link: string,
+   position: number,
+ }
+
 const searchapi:ScraperSettings = {
  id: 'searchapi',
  name: 'SearchApi.io',
  website: 'searchapi.io',
+  allowsCity: true,
  headers: (keyword, settings) => {
     return {
        'Content-Type': 'application/json',
@@ -9,7 +18,10 @@ const searchapi:ScraperSettings = {
     };
  },
  scrapeURL: (keyword) => {
-     return `https://www.searchapi.io/api/v1/search?engine=google&q=${encodeURI(keyword.keyword)}&num=100&gl=${keyword.country}&device=${keyword.device}`;
+   const country = keyword.country || 'US';
+   const countryName = countries[country][0];
+   const location = keyword.city && countryName ? `&location=${encodeURI(`${keyword.city},${countryName}`)}` : '';
+     return `https://www.searchapi.io/api/v1/search?engine=google&q=${encodeURI(keyword.keyword)}&num=100&gl=${country}&device=${keyword.device}${location}`;
  },
  resultObjectKey: 'organic_results',
  serpExtractor: (content) => {
@@ -29,10 +41,4 @@ const searchapi:ScraperSettings = {
  },
 };

-interface SearchApiResult {
-  title: string,
-  link: string,
-  position: number,
-}
-
 export default searchapi;
--- a/scrapers/services/serpapi.ts
+++ b/scrapers/services/serpapi.ts
@@ -1,3 +1,5 @@
+import countries from '../../utils/countries';
+
 interface SerpApiResult {
   title: string,
   link: string,
@@ -8,6 +10,7 @@ const serpapi:ScraperSettings = {
   id: 'serpapi',
   name: 'SerpApi.com',
   website: 'serpapi.com',
+   allowsCity: true,
   headers: (keyword, settings) => {
      return {
         'Content-Type': 'application/json',
@@ -15,7 +18,9 @@ const serpapi:ScraperSettings = {
      };
   },
   scrapeURL: (keyword, settings) => {
-      return `https://serpapi.com/search?q=${encodeURI(keyword.keyword)}&num=100&gl=${keyword.country}&device=${keyword.device}&api_key=${settings.scaping_api}`;
+      const countryName = countries[keyword.country || 'US'][0];
+      const location = keyword.city && keyword.country ? `&location=${encodeURI(`${keyword.city},${countryName}`)}` : '';
+      return `https://serpapi.com/search?q=${encodeURI(keyword.keyword)}&num=100&gl=${keyword.country}&device=${keyword.device}${location}&api_key=${settings.scaping_api}`;
   },
   resultObjectKey: 'organic_results',
   serpExtractor: (content) => {
--- a/scrapers/services/serper.ts
+++ b/scrapers/services/serper.ts
@@ -0,0 +1,35 @@
+interface SerperResult {
+   title: string,
+   link: string,
+   position: number,
+}
+
+const serper:ScraperSettings = {
+   id: 'serper',
+   name: 'Serper.dev',
+   website: 'serper.dev',
+   allowsCity: true,
+   scrapeURL: (keyword, settings, countryData) => {
+      const country = keyword.country || 'US';
+      const lang = countryData[country][2];
+      return `https://google.serper.dev/search?q=${encodeURI(keyword.keyword)}&gl=${country}&hl=${lang}&num=100&apiKey=${settings.scaping_api}`;
+   },
+   resultObjectKey: 'organic',
+   serpExtractor: (content) => {
+      const extractedResult = [];
+      const results: SerperResult[] = (typeof content === 'string') ? JSON.parse(content) : content as SerperResult[];
+
+      for (const { link, title, position } of results) {
+         if (title && link) {
+            extractedResult.push({
+               title,
+               url: link,
+               position,
+            });
+         }
+      }
+      return extractedResult;
+   },
+};
+
+export default serper;
--- a/scrapers/services/spaceserp.ts
+++ b/scrapers/services/spaceserp.ts
@@ -1,3 +1,5 @@
+import countries from '../../utils/countries';
+
 interface SpaceSerpResult {
   title: string,
   link: string,
@@ -9,10 +11,14 @@ const spaceSerp:ScraperSettings = {
   id: 'spaceSerp',
   name: 'Space Serp',
   website: 'spaceserp.com',
+   allowsCity: true,
   scrapeURL: (keyword, settings, countryData) => {
      const country = keyword.country || 'US';
+      const countryName = countries[country][0];
+      const location = keyword.city ? `&location=${encodeURI(`${keyword.city},${countryName}`)}` : '';
+      const device = keyword.device === 'mobile' ? '&device=mobile' : '';
      const lang = countryData[country][2];
-      return `https://api.spaceserp.com/google/search?apiKey=${settings.scaping_api}&q=${encodeURI(keyword.keyword)}&pageSize=100&gl=${country}&hl=${lang}${keyword.device === 'mobile' ? '&device=mobile' : ''}&resultBlocks=`;
+      return `https://api.spaceserp.com/google/search?apiKey=${settings.scaping_api}&q=${encodeURI(keyword.keyword)}&pageSize=100&gl=${country}&hl=${lang}${location}${device}&resultBlocks=`;
   },
   resultObjectKey: 'organic_results',
   serpExtractor: (content) => {
--- a/scrapers/services/valueserp.ts
+++ b/scrapers/services/valueserp.ts
@@ -0,0 +1,41 @@
+import countries from '../../utils/countries';
+
+interface ValueSerpResult {
+   title: string,
+   link: string,
+   position: number,
+   domain: string,
+}
+
+const valueSerp:ScraperSettings = {
+   id: 'valueserp',
+   name: 'Value Serp',
+   website: 'valueserp.com',
+   allowsCity: true,
+   scrapeURL: (keyword, settings, countryData) => {
+      const country = keyword.country || 'US';
+      const countryName = countries[country][0];
+      const location = keyword.city ? `&location=${encodeURI(`${keyword.city},${countryName}`)}` : '';
+      const device = keyword.device === 'mobile' ? '&device=mobile' : '';
+      const lang = countryData[country][2];
+      console.log(`https://api.valueserp.com/search?api_key=${settings.scaping_api}&q=${encodeURI(keyword.keyword)}&gl=${country}&hl=${lang}${device}${location}&num=100&output=json&include_answer_box=false&include_advertiser_info=false`);
+      return `https://api.valueserp.com/search?api_key=${settings.scaping_api}&q=${encodeURI(keyword.keyword)}&gl=${country}&hl=${lang}${device}${location}&num=100&output=json&include_answer_box=false&include_advertiser_info=false`;
+   },
+   resultObjectKey: 'organic_results',
+   serpExtractor: (content) => {
+      const extractedResult = [];
+      const results: ValueSerpResult[] = (typeof content === 'string') ? JSON.parse(content) : content as ValueSerpResult[];
+      for (const result of results) {
+         if (result.title && result.link) {
+            extractedResult.push({
+               title: result.title,
+               url: result.link,
+               position: result.position,
+            });
+         }
+      }
+      return extractedResult;
+   },
+};
+
+export default valueSerp;
--- a/services/domains.tsx
+++ b/services/domains.tsx
@@ -19,6 +19,19 @@ export async function fetchDomains(router: NextRouter, withStats:boolean): Promi
   return res.json();
 }

+export async function fetchDomain(router: NextRouter, domainName: string): Promise<{domain: DomainType}> {
+   if (!domainName) { throw new Error('No Domain Name Provided!'); }
+   const res = await fetch(`${window.location.origin}/api/domain?domain=${domainName}`, { method: 'GET' });
+   if (res.status >= 400 && res.status < 600) {
+      if (res.status === 401) {
+         console.log('Unauthorized!!');
+         router.push('/login');
+      }
+      throw new Error('Bad response from server');
+   }
+   return res.json();
+}
+
 export async function fetchDomainScreenshot(domain: string, screenshot_key:string, forceFetch = false): Promise<string | false> {
   const domainThumbsRaw = localStorage.getItem('domainThumbs');
   const domThumbs = domainThumbsRaw ? JSON.parse(domainThumbsRaw) : {};
@@ -53,6 +66,14 @@ export function useFetchDomains(router: NextRouter, withStats:boolean = false) {
   return useQuery('domains', () => fetchDomains(router, withStats));
 }

+export function useFetchDomain(router: NextRouter, domainName:string, onSuccess: Function) {
+   return useQuery('domain', () => fetchDomain(router, domainName), {
+      onSuccess: async (data) => {
+         console.log('Domain Loaded!!!', data.domain);
+         onSuccess(data.domain);
+      } });
+}
+
 export function useAddDomain(onSuccess:Function) {
   const router = useRouter();
   const queryClient = useQueryClient();
@@ -89,10 +110,11 @@ export function useUpdateDomain(onSuccess:Function) {
      const headers = new Headers({ 'Content-Type': 'application/json', Accept: 'application/json' });
      const fetchOpts = { method: 'PUT', headers, body: JSON.stringify(domainSettings) };
      const res = await fetch(`${window.location.origin}/api/domains?domain=${domain.domain}`, fetchOpts);
+      const responseObj = await res.json();
      if (res.status >= 400 && res.status < 600) {
-         throw new Error('Bad response from server');
+         throw new Error(responseObj?.error || 'Bad response from server');
      }
-      return res.json();
+      return responseObj;
   }, {
      onSuccess: async () => {
         console.log('Settings Updated!!!');
@@ -100,8 +122,8 @@ export function useUpdateDomain(onSuccess:Function) {
         onSuccess();
         queryClient.invalidateQueries(['domains']);
      },
-      onError: () => {
-         console.log('Error Updating Domain Settings!!!');
+      onError: (error) => {
+         console.log('Error Updating Domain Settings!!!', error);
         toast('Error Updating Domain Settings', { icon: '⚠️' });
      },
   });
--- a/services/keywords.tsx
+++ b/services/keywords.tsx
@@ -2,16 +2,21 @@ import toast from 'react-hot-toast';
 import { NextRouter } from 'next/router';
 import { useMutation, useQuery, useQueryClient } from 'react-query';

-export const fetchKeywords = async (router: NextRouter) => {
-   if (!router.query.slug) { return []; }
-   const res = await fetch(`${window.location.origin}/api/keywords?domain=${router.query.slug}`, { method: 'GET' });
+export const fetchKeywords = async (router: NextRouter, domain: string) => {
+   if (!domain) { return []; }
+   const res = await fetch(`${window.location.origin}/api/keywords?domain=${domain}`, { method: 'GET' });
   return res.json();
 };

-export function useFetchKeywords(router: NextRouter, setKeywordSPollInterval?:Function, keywordSPollInterval:undefined|number = undefined) {
+export function useFetchKeywords(
+   router: NextRouter,
+   domain: string,
+   setKeywordSPollInterval?:Function,
+   keywordSPollInterval:undefined|number = undefined,
+) {
   const { data: keywordsData, isLoading: keywordsLoading, isError } = useQuery(
-      ['keywords', router.query.slug],
-      () => fetchKeywords(router),
+      ['keywords', domain],
+      () => fetchKeywords(router, domain),
      {
         refetchInterval: keywordSPollInterval,
         onSuccess: (data) => {
--- a/services/settings.ts
+++ b/services/settings.ts
@@ -60,3 +60,37 @@ export function useClearFailedQueue(onSuccess:Function) {
      },
   });
 }
+
+export async function fetchMigrationStatus() {
+   const res = await fetch(`${window.location.origin}/api/dbmigrate`, { method: 'GET' });
+   return res.json();
+}
+
+export function useCheckMigrationStatus() {
+   return useQuery('dbmigrate', () => fetchMigrationStatus());
+}
+
+export const useMigrateDatabase = (onSuccess:Function|undefined) => {
+   const queryClient = useQueryClient();
+
+   return useMutation(async () => {
+      // console.log('settings: ', JSON.stringify(settings));
+      const res = await fetch(`${window.location.origin}/api/dbmigrate`, { method: 'POST' });
+      if (res.status >= 400 && res.status < 600) {
+         throw new Error('Bad response from server');
+      }
+      return res.json();
+   }, {
+      onSuccess: async (res) => {
+         if (onSuccess) {
+            onSuccess(res);
+         }
+         toast('Database Updated!', { icon: '✔️' });
+         queryClient.invalidateQueries(['settings']);
+      },
+      onError: () => {
+         console.log('Error Updating Database!!!');
+         toast('Error Updating Database.', { icon: '⚠️' });
+      },
+   });
+};
--- a/types.d.ts
+++ b/types.d.ts
@@ -15,6 +15,7 @@ type DomainType = {
   scVisits?: number,
   scImpressions?: number,
   scPosition?: number,
+   search_console?: string,
 }

 type KeywordHistory = {
@@ -39,6 +40,7 @@ type KeywordType = {
   lastUpdateError: {date: string, error: string, scraper: string} | false,
   scData?: KeywordSCData,
   uid?: string
+   city?: string
 }

 type KeywordLastResult = {
@@ -61,9 +63,17 @@ type countryCodeData = {
   [ISO:string] : string
 }

+type DomainSearchConsole = {
+   property_type: 'domain' | 'url',
+   url: string,
+   client_email:string,
+   private_key:string,
+}
+
 type DomainSettings = {
   notification_interval: string,
   notification_emails: string,
+   search_console?: DomainSearchConsole
 }

 type SettingsType = {
@@ -77,14 +87,17 @@ type SettingsType = {
   smtp_port: string,
   smtp_username?: string,
   smtp_password?: string,
-   search_console_integrated?: boolean,
-   available_scapers?: Array,
+   available_scapers?: { label: string, value: string, allowsCity?: boolean }[],
   scrape_interval?: string,
   scrape_delay?: string,
   scrape_retry?: boolean,
   failed_queue?: string[]
   version?: string,
   screenshot_key?: string,
+   search_console: boolean,
+   search_console_client_email: string,
+   search_console_private_key: string,
+   search_console_integrated?: boolean,
 }

 type KeywordSCDataChild = {
@@ -108,7 +121,8 @@ type KeywordAddPayload = {
   device: string,
   country: string,
   domain: string,
-   tags: string,
+   tags?: string,
+   city?:string
 }

 type SearchAnalyticsRawItem = {
@@ -177,11 +191,23 @@ type scraperExtractedItem = {
   position: number,
 }
 interface ScraperSettings {
+   /** A Unique ID for the Scraper. eg: myScraper */
   id:string,
+   /** The Name of the Scraper */
   name:string,
+   /** The Website address of the Scraper */
   website:string,
+   /** The result object's key that contains the results of the scraped data. For example,
+    * if your scraper API the data like this `{scraped:[item1,item2..]}` the resultObjectKey should be "scraped" */
   resultObjectKey: string,
+   /** If the Scraper allows setting a perices location or allows city level scraping set this to true. */
+   allowsCity?: boolean,
+   /** Set your own custom HTTP header properties when making the scraper API request.
+    * The function should return an object that contains all the header properties you want to pass to API request's header.
+    * Example: `{'Cache-Control': 'max-age=0', 'Content-Type': 'application/json'}` */
   headers?(keyword:KeywordType, settings: SettingsType): Object,
+   /** Construct the API URL for scraping the data through your Scraper's API */
   scrapeURL?(keyword:KeywordType, settings:SettingsType, countries:countryData): string,
+   /** Custom function to extract the serp result from the scraped data. The extracted data should be @return {scraperExtractedItem[]} */
   serpExtractor?(content:string): scraperExtractedItem[],
 }
--- a/utils/client/validators.ts
+++ b/utils/client/validators.ts
@@ -33,3 +33,14 @@ export const isValidDomain = (domain:string): boolean => {

   return isValid;
 };
+
+export const isValidUrl = (str: string) => {
+   let url;
+
+   try {
+     url = new URL(str);
+   } catch (e) {
+     return false;
+   }
+   return url.protocol === 'http:' || url.protocol === 'https:';
+ };
--- a/utils/scraper.ts
+++ b/utils/scraper.ts
@@ -187,15 +187,17 @@ export const extractScrapedResult = (content: string, device: string): SearchRes

 /**
 * Find in the domain's position from the extracted search result.
- * @param {string} domain - Domain Name to look for.
+ * @param {string} domainURL - URL Name to look for.
 * @param {SearchResult[]} result - The search result array extracted from the Google Search result.
 * @returns {SERPObject}
 */
-export const getSerp = (domain:string, result:SearchResult[]) : SERPObject => {
-   if (result.length === 0 || !domain) { return { postion: 0, url: '' }; }
+export const getSerp = (domainURL:string, result:SearchResult[]) : SERPObject => {
+   if (result.length === 0 || !domainURL) { return { postion: 0, url: '' }; }
+   const URLToFind = new URL(domainURL.includes('https://') ? domainURL : `https://${domainURL}`);
+   const theURL = URLToFind.hostname + URLToFind.pathname;
   const foundItem = result.find((item) => {
-      const itemDomain = item.url.replace('www.', '').match(/^(?:https?:)?(?:\/\/)?([^/?]+)/i);
-      return itemDomain && itemDomain.includes(domain.replace('www.', ''));
+      const itemURL = new URL(item.url.includes('https://') ? item.url : `https://${item.url}`);
+      return theURL === itemURL.hostname + itemURL.pathname || `${theURL}/` === itemURL.hostname + itemURL.pathname;
   });
   return { postion: foundItem ? foundItem.position : 0, url: foundItem && foundItem.url ? foundItem.url : '' };
 };
--- a/utils/searchConsole.ts
+++ b/utils/searchConsole.ts
@@ -1,4 +1,5 @@
 import { auth, searchconsole_v1 } from '@googleapis/searchconsole';
+import Cryptr from 'cryptr';
 import { readFile, writeFile, unlink } from 'fs/promises';
 import { getCountryCodeFromAlphaThree } from './countries';

@@ -7,22 +8,32 @@ export type SCDomainFetchError = {
   errorMsg: string,
 }

+type SCAPISettings = { client_email: string, private_key: string }
+
 type fetchConsoleDataResponse = SearchAnalyticsItem[] | SearchAnalyticsStat[] | SCDomainFetchError;

 /**
- * function that retrieves data from the Google Search Console API based on the provided domain name, number of days, and optional type.
- * @param {string} domainName - The domain name for which you want to fetch search console data.
- * @param {number} days - The `days` parameter is the number of days of data you want to fetch from the Search Console.
- * @param {string} [type] - The `type` parameter is an optional parameter that specifies the type of data to fetch from the Search Console API.
+ * Retrieves data from the Google Search Console API based on the provided domain name, number of days, and optional type.
+ * @param {DomainType} domain - The domain for which you want to fetch search console data.
+ * @param {number} days - number of days of data you want to fetch from the Search Console.
+ * @param {string} [type] - (optional) specifies the type of data to fetch from the Search Console.
+ * @param {SCAPISettings} [api] - (optional) specifies the Seach Console API Information.
 * @returns {Promise<fetchConsoleDataResponse>}
 */
-const fetchSearchConsoleData = async (domainName:string, days:number, type?:string): Promise<fetchConsoleDataResponse> => {
-   if (!domainName) return { error: true, errorMsg: 'Domain Not Provided!' };
+const fetchSearchConsoleData = async (domain:DomainType, days:number, type?:string, api?:SCAPISettings): Promise<fetchConsoleDataResponse> => {
+   if (!domain) return { error: true, errorMsg: 'Domain Not Provided!' };
+   if (!api?.private_key || !api?.client_email) return { error: true, errorMsg: 'Search Console API Data Not Avaialable.' };
+   const domainName = domain.domain;
+   const defaultSCSettings = { property_type: 'domain', url: '', client_email: '', private_key: '' };
+   const domainSettings = domain.search_console ? JSON.parse(domain.search_console) : defaultSCSettings;
+   const sCPrivateKey = api?.private_key || process.env.SEARCH_CONSOLE_PRIVATE_KEY || '';
+   const sCClientEmail = api?.client_email || process.env.SEARCH_CONSOLE_CLIENT_EMAIL || '';
+
   try {
   const authClient = new auth.GoogleAuth({
      credentials: {
-        private_key: process.env.SEARCH_CONSOLE_PRIVATE_KEY ? process.env.SEARCH_CONSOLE_PRIVATE_KEY.replaceAll('\\n', '\n') : '',
-        client_email: process.env.SEARCH_CONSOLE_CLIENT_EMAIL ? process.env.SEARCH_CONSOLE_CLIENT_EMAIL : '',
+        private_key: (sCPrivateKey).replaceAll('\\n', '\n'),
+        client_email: (sCClientEmail || '').trim(),
      },
      scopes: [
        'https://www.googleapis.com/auth/webmasters.readonly',
@@ -51,7 +62,8 @@ const fetchSearchConsoleData = async (domainName:string, days:number, type?:stri
      };
   }

-      const res = client.searchanalytics.query({ siteUrl: `sc-domain:${domainName}`, requestBody });
+      const siteUrl = domainSettings.property_type === 'url' && domainSettings.url ? domainSettings.url : `sc-domain:${domainName}`;
+      const res = client.searchanalytics.query({ siteUrl, requestBody });
      const resData:any = (await res).data;
      let finalRows = resData.rows ? resData.rows.map((item:SearchAnalyticsRawItem) => parseSearchConsoleItem(item, domainName)) : [];

@@ -70,26 +82,29 @@ const fetchSearchConsoleData = async (domainName:string, days:number, type?:stri
      }

      return finalRows;
-   } catch (error:any) {
+   } catch (err:any) {
      const qType = type === 'stats' ? '(stats)' : `(${days}days)`;
-      console.log(`[ERROR] Search Console API Error for ${domainName} ${qType} : `, error?.response?.status, error?.response?.statusText);
-      return { error: true, errorMsg: `${error?.response?.status}: ${error?.response?.statusText}` };
+      const errorMsg = err?.response?.status && `${err?.response?.statusText}. ${err?.response?.data?.error_description}`;
+      console.log(`[ERROR] Search Console API Error for ${domainName} ${qType} : `, errorMsg || err?.code);
+      // console.log('SC ERROR :', err);
+      return { error: true, errorMsg: errorMsg || err?.code };
   }
 };

 /**
 * The function fetches search console data for a given domain and returns it in a structured format.
- * @param {string} domain - The `domain` parameter is a string that represents the domain for which we
+ * @param {DomainType} domain - The `domain` parameter is a Domain object that represents the domain for which we
 * want to fetch search console data.
 * @returns The function `fetchDomainSCData` is returning a Promise that resolves to an object of type
 * `SCDomainDataType`.
 */
-export const fetchDomainSCData = async (domain:string): Promise<SCDomainDataType> => {
+export const fetchDomainSCData = async (domain:DomainType, scAPI?: SCAPISettings): Promise<SCDomainDataType> => {
   const days = [3, 7, 30];
   const scDomainData:SCDomainDataType = { threeDays: [], sevenDays: [], thirtyDays: [], lastFetched: '', lastFetchError: '', stats: [] };
-   if (domain) {
+   if (domain.domain && scAPI) {
+      const theDomain = domain;
      for (const day of days) {
-         const items = await fetchSearchConsoleData(domain, day);
+         const items = await fetchSearchConsoleData(theDomain, day, undefined, scAPI);
          scDomainData.lastFetched = new Date().toJSON();
         if (Array.isArray(items)) {
            if (day === 3) scDomainData.threeDays = items as SearchAnalyticsItem[];
@@ -99,11 +114,11 @@ export const fetchDomainSCData = async (domain:string): Promise<SCDomainDataType
            scDomainData.lastFetchError = items.errorMsg;
         }
      }
-      const stats = await fetchSearchConsoleData(domain, 30, 'stat');
+      const stats = await fetchSearchConsoleData(theDomain, 30, 'stat', scAPI);
      if (stats && Array.isArray(stats) && stats.length > 0) {
         scDomainData.stats = stats as SearchAnalyticsStat[];
      }
-      await updateLocalSCData(domain, scDomainData);
+      await updateLocalSCData(domain.domain, scDomainData);
   }

   return scDomainData;
@@ -139,9 +154,9 @@ export const integrateKeywordSCData = (keyword: KeywordType, SCData:SCDomainData
   const ctr:any = { yesterday: 0, threeDays: 0, sevenDays: 0, thirtyDays: 0, avgSevenDays: 0, avgThreeDays: 0, avgThirtyDays: 0 };
   const position:any = { yesterday: 0, threeDays: 0, sevenDays: 0, thirtyDays: 0, avgSevenDays: 0, avgThreeDays: 0, avgThirtyDays: 0 };

-   const threeDaysData = SCData.threeDays.find((item:SearchAnalyticsItem) => item.uid === kuid) || {};
-   const SevenDaysData = SCData.sevenDays.find((item:SearchAnalyticsItem) => item.uid === kuid) || {};
-   const ThirdyDaysData = SCData.thirtyDays.find((item:SearchAnalyticsItem) => item.uid === kuid) || {};
+   const threeDaysData = SCData?.threeDays?.find((item:SearchAnalyticsItem) => item.uid === kuid) || {};
+   const SevenDaysData = SCData?.sevenDays?.find((item:SearchAnalyticsItem) => item.uid === kuid) || {};
+   const ThirdyDaysData = SCData?.thirtyDays?.find((item:SearchAnalyticsItem) => item.uid === kuid) || {};
   const totalData:any = { threeDays: threeDaysData, sevenDays: SevenDaysData, thirtyDays: ThirdyDaysData };

   Object.keys(totalData).forEach((dataKey) => {
@@ -164,16 +179,71 @@ export const integrateKeywordSCData = (keyword: KeywordType, SCData:SCDomainData
   return { ...keyword, scData: finalSCData };
 };

+/**
+ * Retrieves the Search Console API information for a given domain.
+ * @param {DomainType} domain - The `domain` parameter is of type `DomainType`, which represents a
+ * domain object. It likely contains information about a specific domain, such as its name, search
+ * console settings, etc.
+ * @returns an object of type `SCAPISettings`.
+ */
+export const getSearchConsoleApiInfo = async (domain: DomainType): Promise<SCAPISettings> => {
+   const scAPIData = { client_email: '', private_key: '' };
+   // Check if the Domain Has the API Data
+   const domainSCSettings = domain.search_console && JSON.parse(domain.search_console);
+   if (domainSCSettings && domainSCSettings.private_key) {
+      if (!domainSCSettings.private_key.includes('BEGIN PRIVATE KEY')) {
+         const cryptr = new Cryptr(process.env.SECRET as string);
+         scAPIData.client_email = domainSCSettings.client_email ? cryptr.decrypt(domainSCSettings.client_email) : '';
+         scAPIData.private_key = domainSCSettings.private_key ? cryptr.decrypt(domainSCSettings.private_key) : '';
+      } else {
+         scAPIData.client_email = domainSCSettings.client_email;
+         scAPIData.private_key = domainSCSettings.private_key;
+      }
+   }
+   // Check if the App Settings Has the API Data
+   if (!scAPIData?.private_key) {
+      const settingsRaw = await readFile(`${process.cwd()}/data/settings.json`, { encoding: 'utf-8' });
+      const settings: SettingsType = settingsRaw ? JSON.parse(settingsRaw) : {};
+      const cryptr = new Cryptr(process.env.SECRET as string);
+      scAPIData.client_email = settings.search_console_client_email ? cryptr.decrypt(settings.search_console_client_email) : '';
+      scAPIData.private_key = settings.search_console_private_key ? cryptr.decrypt(settings.search_console_private_key) : '';
+   }
+   if (!scAPIData?.private_key && process.env.SEARCH_CONSOLE_PRIVATE_KEY && process.env.SEARCH_CONSOLE_CLIENT_EMAIL) {
+      scAPIData.client_email = process.env.SEARCH_CONSOLE_CLIENT_EMAIL;
+      scAPIData.private_key = process.env.SEARCH_CONSOLE_PRIVATE_KEY;
+   }
+
+   return scAPIData;
+};
+
+/**
+ * Checks if the provided domain level Google Search Console API info is valid.
+ * @param {DomainType} domain - The domain that represents the domain for which the SC API info is being checked.
+ * @returns an object of type `{ isValid: boolean, error: string }`.
+ */
+export const checkSerchConsoleIntegration = async (domain: DomainType): Promise<{ isValid: boolean, error: string }> => {
+   const res = { isValid: false, error: '' };
+   const { client_email = '', private_key = '' } = domain?.search_console ? JSON.parse(domain.search_console) : {};
+   const response = await fetchSearchConsoleData(domain, 3, undefined, { client_email, private_key });
+   if (Array.isArray(response)) { res.isValid = true; }
+   if ((response as SCDomainFetchError)?.errorMsg) { res.error = (response as SCDomainFetchError).errorMsg; }
+   return res;
+};
+
 /**
 * The function reads and returns the domain-specific data stored in a local JSON file.
 * @param {string} domain - The `domain` parameter is a string that represents the domain for which the SC data is being read.
 * @returns {Promise<SCDomainDataType>}
 */
-export const readLocalSCData = async (domain:string): Promise<SCDomainDataType> => {
-   const filePath = `${process.cwd()}/data/SC_${domain}.json`;
-   const currentQueueRaw = await readFile(filePath, { encoding: 'utf-8' }).catch(async () => { await updateLocalSCData(domain); return '{}'; });
-   const domainSCData = JSON.parse(currentQueueRaw);
-   return domainSCData;
+export const readLocalSCData = async (domain:string): Promise<SCDomainDataType|false> => {
+   try {
+      const filePath = `${process.cwd()}/data/SC_${domain.replaceAll('/', '-')}.json`;
+      const currentQueueRaw = await readFile(filePath, { encoding: 'utf-8' }).catch(async () => { await updateLocalSCData(domain); return '{}'; });
+      const domainSCData = JSON.parse(currentQueueRaw);
+      return domainSCData;
+   } catch (error) {
+      return false;
+   }
 };

 /**
@@ -183,10 +253,14 @@ export const readLocalSCData = async (domain:string): Promise<SCDomainDataType>
 * @returns {Promise<SCDomainDataType|false>}
 */
 export const updateLocalSCData = async (domain:string, scDomainData?:SCDomainDataType): Promise<SCDomainDataType|false> => {
-   const filePath = `${process.cwd()}/data/SC_${domain}.json`;
-   const emptyData:SCDomainDataType = { threeDays: [], sevenDays: [], thirtyDays: [], lastFetched: '', lastFetchError: '' };
-   await writeFile(filePath, JSON.stringify(scDomainData || emptyData), { encoding: 'utf-8' }).catch((err) => { console.log(err); });
-   return scDomainData || emptyData;
+   try {
+      const filePath = `${process.cwd()}/data/SC_${domain.replaceAll('/', '-')}.json`;
+      const emptyData:SCDomainDataType = { threeDays: [], sevenDays: [], thirtyDays: [], lastFetched: '', lastFetchError: '' };
+      await writeFile(filePath, JSON.stringify(scDomainData || emptyData), { encoding: 'utf-8' }).catch((err) => { console.log(err); });
+      return scDomainData || emptyData;
+   } catch (error) {
+      return false;
+   }
 };

 /**
@@ -195,7 +269,7 @@ export const updateLocalSCData = async (domain:string, scDomainData?:SCDomainDat
 * @returns {Promise<boolean>} - Returns true if file was removed, else returns false.
 */
 export const removeLocalSCData = async (domain:string): Promise<boolean> => {
-   const filePath = `${process.cwd()}/data/SC_${domain}.json`;
+   const filePath = `${process.cwd()}/data/SC_${domain.replaceAll('/', '-')}.json`;
   try {
      await unlink(filePath);
      return true;
Author	SHA1	Message	Date
towfiqi	e5ad7a3175	chore(release): 1.0.1	2024-02-13 23:55:32 +06:00
towfiqi	e5dd411aa9	fix: Resolves the app crash issue when there is no database. closes #161, #162	2024-02-13 23:54:55 +06:00
towfiqi	c3ddb9d3c3	chore(release): 1.0.0	2024-02-09 21:47:39 +06:00
towfiqi	dbf540cfdb	fix: Resolves missing Keyword Loading Spinner issue.	2024-02-09 21:18:23 +06:00
towfiqi	1f0831ed13	chore: Updates vulnerable dependencies.	2024-02-09 00:56:59 +06:00
towfiqi	b4ad69baaa	feat: Adds Serper.dev integration closes #138	2024-02-09 00:43:28 +06:00
towfiqi	f04b10cf6b	feat: Adds the ability to setup Search Console through the UI. - Adds the ability to add domain specific Search Console API Info through the Domain Settings panel. - Adds the ability to add global Search Console API Info through the App Settings Panel. - Adds better Search Console Error logging. - Changes the App Settings Sidebar UI. - Changers the Domain Settings Modal UI. - Replaces html Input field with custom InputField component. - Adds a new /domain api route to get the full domain info which includes the domain level Search console API. closes #59, #146	2024-02-08 22:14:24 +06:00
towfiqi	b2e97b2ebe	feat: Adds the Ability to set Search Console Property type via Domain Settings. - Previously only domain properties worked with SerpBear. This feature adds the ability to add URL properties as well. - Adds a new field "search_console" in Domain Table. - Adds a new Search Console option in Domain Settings Modal UI. - When the new "This is a URL Property" option is enabled, the exact Property URL should be provided. closes #50	2024-02-06 23:42:28 +06:00
towfiqi	1041cb3c0b	feat: Adds ValueSerp Integration. closes #105, #106	2024-02-06 13:32:24 +06:00
towfiqi	3719f21d98	feat: Adds the ability for city level scraping for scapers that allow it. - Only available for scrapers that allows custom location or city level scraping. - When a city level keyword is added the city name is displayed in the keyword title. closes #139, #151	2024-02-06 13:22:32 +06:00
towfiqi	444ba5d461	refactor: Adds Keyword Table migration to add new fields. - Adds city, latlong and settings fields to Keyword table.	2024-02-04 23:43:13 +06:00
towfiqi	dd54e535c9	build: adds database migration method.	2024-02-04 23:39:39 +06:00
towfiqi	34d121dac7	refactor: removes unnecessary useEffect	2024-02-04 10:27:44 +06:00
towfiqi	3c2a1b8a5b	feat: adds the ability to add url as a domain. You can now track specific marketplace/social domain URLs. For example a reddit.com post, an amazon.com product, github repo etc. closes: #53, #90, #119	2024-02-03 20:17:51 +06:00
towfiqi	e2ecdef10e	Chore: Fixes Typo	2024-02-03 10:15:07 +06:00
towfiqi	633ab2c467	fix: Resolves Keywords filter crashing issue.	2024-02-03 10:14:46 +06:00