Python: ssenariysi Google Autosuggest-ni qidirish kalit so'zlari uchun trendlar ko'chirmasi

Autosuggest tendentsiyalarini qo'lga kiritish uchun Python skript

Google Trends-ni hamma yoqtiradi, ammo Long Tail kalit so'zlari haqida gap ketganda, bu biroz hiyla-nayrang. Biz hammamiz rasmiyni yoqtiramiz google trend xizmati qidiruv xatti-harakatlari to'g'risida tushunchalar olish uchun. Biroq, ikkita narsa ko'pchilik uni qattiq ish uchun ishlatishiga xalaqit beradi;

  1. Qachon topishingiz kerak bo'lsa yangi joy kalit so'zlari, Mana Google Trends ma'lumotlari etarli emas 
  2. Google trendlariga so'rov yuborish uchun rasmiy API etishmasligi: shunga o'xshash modullardan foydalanganda pytrends, keyin proksi-serverlardan foydalanishimiz kerak, yoki biz bloklanib qolamiz. 

Ushbu maqolada Google Autosuggest orqali ommabop kalit so'zlarni eksport qilish uchun biz yozgan Python skriptini baham ko'raman.

Vaqt o'tishi bilan Autosuggest natijalarini oling va saqlang 

Google Autosuggest-ga yuborish uchun bizda 1,000 ta Seed kalit so'zlari bor deb taxmin qiling. Buning evaziga, ehtimol, 200,000 ming atrofida olamiz uzun quyruq kalit so'zlar. Keyin, biz bir hafta o'tgach, xuddi shu narsani qilishimiz va ikkita savolga javob berish uchun ushbu ma'lumotlar to'plamlarini taqqoslashimiz kerak:

  • Qaysi so'rovlar yangi kalit so'zlar oxirgi marta solishtirganda? Bu, ehtimol, bizga kerak bo'lgan holat. Google ushbu so'rovlar ahamiyatli bo'lib bormoqda, deb o'ylaydi - shu bilan biz o'zimizning Google Autosuggest echimini yaratishimiz mumkin! 
  • Qaysi so'rovlar kalit so'zlar endi yo'q trendmi?

Ssenariy juda oson va kodning aksariyati men bilan o'rtoqlashdi Bu yerga. Yangilangan kod o'tgan ish vaqtidagi ma'lumotlarni saqlaydi va vaqt o'tishi bilan takliflarni taqqoslaydi. Biz buni sodda qilish uchun SQLite kabi fayllarga asoslangan ma'lumotlar bazalaridan qochdik - shuning uchun barcha ma'lumotlarni saqlash quyidagi CSV fayllaridan foydalanadi. Bu sizga faylni Excelga import qilish va o'zingizning biznesingiz uchun kalit so'z tendentsiyalarini o'rganish imkonini beradi.

Ushbu Python skriptidan foydalanish uchun

  1. Avtomatik to'ldirishga yuborilishi kerak bo'lgan kalit so'zlar to'plamini kiriting: keywords.csv
  2. O'zingizning ehtiyojingiz uchun skript sozlamalarini o'rnating:
    • TIL: standart “en”
    • Mamlakat: standart "biz"
  3. Ssenariyni haftada bir marta ishlashni rejalashtiring. Siz uni xohlaganingizcha qo'lda ishlatishingiz mumkin.
  4. Keyingi tahlil qilish uchun keyword_suggestions.csv dan foydalaning:
    • birinchi_ko'rilgan: bu so'rov birinchi marta avtoulovda paydo bo'lgan sana
    • oxirgi_ko'rilgan: so'rov oxirgi marta ko'rilgan sana
    • yangi: if first_seen == last_seen biz buni o'rnatdik chin - Google autosuggest-da yangi trend qidiruvlarini olish uchun ushbu qiymatni filtrlash kifoya.

Python kodi

# Pemavor.com Autocomplete Trends
# Author: Stefan Neefischer (stefan.neefischer@gmail.com)
import concurrent.futures
from datetime import date
from datetime import datetime
import pandas as pd
import itertools
import requests
import string
import json
import time

charList = " " + string.ascii_lowercase + string.digits

def makeGoogleRequest(query):
    # If you make requests too quickly, you may be blocked by google 
    time.sleep(WAIT_TIME)
    URL="http://suggestqueries.google.com/complete/search"
    PARAMS = {"client":"opera",
            "hl":LANGUAGE,
            "q":query,
            "gl":COUNTRY}
    response = requests.get(URL, params=PARAMS)
    if response.status_code == 200:
        try:
            suggestedSearches = json.loads(response.content.decode('utf-8'))[1]
        except:
            suggestedSearches = json.loads(response.content.decode('latin-1'))[1]
        return suggestedSearches
    else:
        return "ERR"

def getGoogleSuggests(keyword):
    # err_count1 = 0
    queryList = [keyword + " " + char for char in charList]
    suggestions = []
    for query in queryList:
        suggestion = makeGoogleRequest(query)
        if suggestion != 'ERR':
            suggestions.append(suggestion)

    # Remove empty suggestions
    suggestions = set(itertools.chain(*suggestions))
    if "" in suggestions:
        suggestions.remove("")
    return suggestions

def autocomplete(csv_fileName):
    dateTimeObj = datetime.now().date()
    #read your csv file that contain keywords that you want to send to google autocomplete
    df = pd.read_csv(csv_fileName)
    keywords = df.iloc[:,0].tolist()
    resultList = []

    with concurrent.futures.ThreadPoolExecutor(max_workers=MAX_WORKERS) as executor:
        futuresGoogle = {executor.submit(getGoogleSuggests, keyword): keyword for keyword in keywords}

        for future in concurrent.futures.as_completed(futuresGoogle):
            key = futuresGoogle[future]
            for suggestion in future.result():
                resultList.append([key, suggestion])

    # Convert the results to a dataframe
    suggestion_new = pd.DataFrame(resultList, columns=['Keyword','Suggestion'])
    del resultList

    #if we have old results read them
    try:
        suggestion_df=pd.read_csv("keyword_suggestions.csv")
        
    except:
        suggestion_df=pd.DataFrame(columns=['first_seen','last_seen','Keyword','Suggestion'])
    
    suggestionCommon_list=[]
    suggestionNew_list=[]
    for keyword in suggestion_new["Keyword"].unique():
        new_df=suggestion_new[suggestion_new["Keyword"]==keyword]
        old_df=suggestion_df[suggestion_df["Keyword"]==keyword]
        newSuggestion=set(new_df["Suggestion"].to_list())
        oldSuggestion=set(old_df["Suggestion"].to_list())
        commonSuggestion=list(newSuggestion & oldSuggestion)
        new_Suggestion=list(newSuggestion - oldSuggestion)
         
        for suggest in commonSuggestion:
            suggestionCommon_list.append([dateTimeObj,keyword,suggest])
        for suggest in new_Suggestion:
            suggestionNew_list.append([dateTimeObj,dateTimeObj,keyword,suggest])
    
    #new keywords
    newSuggestion_df = pd.DataFrame(suggestionNew_list, columns=['first_seen','last_seen','Keyword','Suggestion'])
    #shared keywords with date update
    commonSuggestion_df = pd.DataFrame(suggestionCommon_list, columns=['last_seen','Keyword','Suggestion'])
    merge=pd.merge(suggestion_df, commonSuggestion_df, left_on=["Suggestion"], right_on=["Suggestion"], how='left')
    merge = merge.rename(columns={'last_seen_y': 'last_seen',"Keyword_x":"Keyword"})
    merge["last_seen"].fillna(merge["last_seen_x"], inplace=True)
    del merge["last_seen_x"]
    del merge["Keyword_y"]
    
    #merge old results with new results
    frames = [merge, newSuggestion_df]
    keywords_df =  pd.concat(frames, ignore_index=True, sort=False)
    # Save dataframe as a CSV file
    keywords_df['first_seen'] = pd.to_datetime(keywords_df['first_seen'])
    keywords_df = keywords_df.sort_values(by=['first_seen','Keyword'], ascending=[False,False])   
    keywords_df['first_seen']= pd.to_datetime(keywords_df['first_seen'])
    keywords_df['last_seen']= pd.to_datetime(keywords_df['last_seen'])
    keywords_df['is_new'] = (keywords_df['first_seen']== keywords_df['last_seen'])
    keywords_df=keywords_df[['first_seen','last_seen','Keyword','Suggestion','is_new']]
    keywords_df.to_csv('keyword_suggestions.csv', index=False)

# If you use more than 50 seed keywords you should slow down your requests - otherwise google is blocking the script
# If you have thousands of seed keywords use e.g. WAIT_TIME = 1 and MAX_WORKERS = 5
WAIT_TIME = 0.2
MAX_WORKERS = 20
# set the autocomplete language
LANGUAGE = "en"
# set the autocomplete country code - DE, US, TR, GR, etc..
COUNTRY="US"
# Keyword_seed csv file name. One column csv file.
#csv_fileName="keyword_seeds.csv"
CSV_FILE_NAME="keywords.csv"
autocomplete(CSV_FILE_NAME)
#The result will save in keyword_suggestions.csv csv file

Python skriptini yuklab oling

Siz nima deb o'ylaysiz?

Ushbu sayt spamni kamaytirish uchun Akismet-dan foydalanadi. Fikringiz ma'lumotlarining qanday ishlashini bilib oling.