CSDN文章搬运至Hexo
下载
我们使用CSDN2md插件:
https://github.com/Qalxry/csdn2md
然后打开我们要下载的专栏
删去并行选型、图片保存本地选型
把间隔时间调整至3秒
添加文件元信息(用于转为hexo)
然后开始下载即可
记得一定要间隔充足的时间!不然会被CSDN识别为爬虫,然后就访问不了了。
转化
我们发现,这样下载后只有一级专栏,二级专栏没了
但我的二级专栏不多,所以可以用Python批量添加:
以一级专栏为 OI 为例,我们想添加二级专栏为 高考复键:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 import reimport yamlfrom pathlib import PathTARGET_CAT = "OI" LEVEL1 = "OI" LEVEL2 = "高考复键" def process_md (file_path: Path ): text = file_path.read_text(encoding="utf-8" ) front_re = re.compile (r"^---\n(.*?)\n---\n" , re.DOTALL) match = front_re.match (text) if not match : print (f"跳过(无头部):{file_path} " ) return fm_raw = match .group(1 ) body = text[match .end():] try : fm_data = yaml.safe_load(fm_raw) except Exception as e: print (f"YAML解析失败 {file_path} :{e} " ) return if fm_data.get("category" ) != TARGET_CAT: return del fm_data["category" ] fm_data["categories" ] = [LEVEL1, LEVEL2] new_fm = yaml.dump( fm_data, allow_unicode=True , sort_keys=False , default_flow_style=False , indent=2 ) full = f"---\n{new_fm} ---\n{body} " file_path.write_text(full, encoding="utf-8" ) print (f"已修改:{file_path} " ) if __name__ == "__main__" : root = Path("." ) md_list = list (root.rglob("*.md" )) print (f"共扫描到 {len (md_list)} 个Markdown文件\n" ) for md in md_list: process_md(md) print ("\n=== 全部处理完成 ===" )
我们转化后的文章如果全放在source中,可能不利于我们后续整理,所以我们可以给它添加前缀
这份脚本添加的是 CSDN算法 的前缀
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 import osprefix = "CSDN算法" for filename in os.listdir("." ): if filename == __file__: continue if os.path.isfile(filename) and filename.lower().endswith(".md" ): new_name = prefix + filename os.rename(filename, new_name) print (f"{filename} → {new_name} " ) print ("\n所有md文件重命名完毕" )
使用脚本:
我们发现,有些时候,爬下来的标签会带"",因此我们可以再写一份Python脚本:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 import osimport redef process_md_file (file_path ): with open (file_path, "r" , encoding="utf-8" ) as f: text = f.read() fm_pattern = re.compile (r"^---\n(.*?)\n---" , re.DOTALL | re.MULTILINE) match = fm_pattern.search(text) if not match : return old_fm_block = match .group(0 ) fm_content = match .group(1 ) lines = fm_content.splitlines() new_lines = [] for line in lines: stripped = line.strip() if re.match (r"^(category|categories|tags):" , stripped): line = re.sub(r'"(.*?)"' , r"\1" , line) elif re.match (r"^\s*-\s*\"" , stripped): line = re.sub(r'"(.*?)"' , r"\1" , line) new_lines.append(line) new_fm_content = "\n" .join(new_lines) new_fm_block = f"---\n{new_fm_content} \n---" new_text = text.replace(old_fm_block, new_fm_block) if new_text != text: with open (file_path, "w" , encoding="utf-8" ) as f: f.write(new_text) print (f"修改完成:{file_path} " ) def scan_md (root="." ): for dirpath, _, files in os.walk(root): for name in files: if name.endswith(".md" ): full_path = os.path.join(dirpath, name) process_md_file(full_path) if __name__ == "__main__" : print ("开始批量移除 front-matter category/tags 双引号..." ) scan_md() print ("全部处理完毕!" )
使用
直接丢到我们的source的_post文件夹中即可