This function is used to scrape text paragraphs from a website.
paragraphs_scrap(
link,
contain = NULL,
case_sensitive = FALSE,
collapse = FALSE,
askRobot = FALSE
)the link of the web page to scrape
filter the paragraphs according to the character string provided.
logical. Should the contain argument be case sensitive ? defaults to FALSE
if TRUE the paragraphs will be collapsed into one element and the contain argument ignored.
logical. Should the function ask the robots.txt if we're allowed or not to scrap the web page ? Default is FALSE.
a character vector.
# \donttest{
# Extracting the paragraphs displayed on the health page of the New York Times
link <- "https://www.nytimes.com/section/health"
paragraphs_scrap(link)# }
#> [1] "Advertisement"
#> [2] "Supported by"
#> [3] "The task force recommends which screenings and other preventive health measures must be covered by insurance."
#> [4] " By Andrew Jacobs"
#> [5] "Doctors and hospitals were subpoenaed for private information on gender-related care for minors, the latest move by the Trump administration to stop the treatments."
#> [6] " By Azeen Ghorayshi and Glenn Thrush"
#> [7] "The letters, many of which were already available online, detail why the regulators initially declined to approve some drugs. All eventually passed muster."
#> [8] " By Christina Jewett and Rebecca Robbins"
#> [9] "Warnings about withdrawal from antidepressants have rippled through society in recent years. A new study claims they are overblown."
#> [10] " By Ellen Barry"
#> [11] "Life Expectancy in California Has Not Rebounded After Covid"
#> [12] "The state’s life expectancy was lower in 2024 than in 2019, according to an analysis, but primarily as a result of causes of death other than Covid."
#> [13] " By Emily Baumgaertner Nunn"
#> [14] "Lung Cancer Screening on Wheels"
#> [15] "On the road with a 68,000-pound tractor-trailer that crisscrosses West Virginia, saving lives."
#> [16] " By Kristian Thacker and Simar Bajaj"
#> [17] "Medical Societies Sue Kennedy and H.H.S. Over Vaccine Advice"
#> [18] "New restrictions on Covid shots run counter to scientific evidence, the groups said."
#> [19] " By Apoorva Mandavilli"
#> [20] "A 37,000-Year Chronicle of What Once Ailed Us"
#> [21] "In a new genetic study, scientists have charted the rise of 214 human diseases across ancient Europe and Asia."
#> [22] " By Carl Zimmer"
#> [23] "A Century Ago, Adolescents Weren’t Fully Human"
#> [24] "Looking back at an awkward moment in the history of adolescent psychology."
#> [25] " By Matt Richtel"
#> [26] "Advertisement"
#> [27] "Measles Cases Hit Highest Total Since U.S. Eliminated the Disease"
#> [28] "Experts worry that if vaccination rates do not improve, deadly outbreaks will become the new normal."
#> [29] " By Teddy Rosenbluth and Jonathan Corum"
#> [30] "Tracking U.S. Measles Outbreaks"
#> [31] "As new cases are reported, our maps show the spread of the virus."
#> [32] " By Jonathan Corum and Teddy Rosenbluth"
#> [33] "What to Know About Measles as the Virus Spreads"
#> [34] "We asked experts about the symptoms of an infection, vaccination and more."
#> [35] " By Dani Blum and Nina Agrawal"
#> [36] "What to Know About Measles When Traveling"
#> [37] "Summer travel raises fears that the highly infectious virus will spread. Here’s how to protect yourself and your family."
#> [38] " By Perri Klass, M.D."
#> [39] "‘I Feel Like I’ve Been Lied To’: When a Measles Outbreak Hits Home"
#> [40] "From a lone clinic in Texas to an entire school district in North Dakota, the virus is upending daily life and revealing a deeper crisis of belief."
#> [41] " By Eli Saslow and Erin Schaff"
#> [42] "Maybe It’s Not Just Aging. Maybe It’s Anemia."
#> [43] "Significant numbers of older people have the condition. Many find relief with an effective treatment that is being more widely prescribed."
#> [44] " By Paula Span"
#> [45] "Many Older People Embrace Vaccines. Research Is Proving Them Right."
#> [46] "Newer formulations are even more effective at preventing illnesses that commonly afflict seniors — perhaps even dementia."
#> [47] " By Paula Span"
#> [48] "Health Risks From Cannabis May Grow as Users Age"
#> [49] "More older people are using cannabis products regularly, but research suggests their cannabis-related health problems are also on the rise."
#> [50] " By Paula Span"
#> [51] "A Ministroke Can Have Major Consequences"
#> [52] "So-called transient ischemic attacks can eventually lead to cognitive declines as steep as those following a full-on stroke, new research finds."
#> [53] " By Paula Span"
#> [54] "Honey, Sweetie, Dearie: The Perils of Elderspeak"
#> [55] "A new training program teaches aides to stop baby talk and address older people as adults."
#> [56] " By Paula Span"
#> [57] "Advertisement"
#> [58] "Readers Share the Unconventional Ways They Grieve"
#> [59] "We asked, and over 800 of you answered."
#> [60] " By Jancee Dunn"
#> [61] "For Some Men, Treating Hair Loss Comes With Fertility Issues"
#> [62] "The medication finasteride has become popular among young men. But for those struggling to conceive, it could be the reason."
#> [63] " By T.M. Brown"
#> [64] "Can ‘Reparenting’ Yourself Make You Happier?"
#> [65] "The concept, centered around healing your ‘inner child,’ is catchy. Here’s what experts have to say."
#> [66] " By Christina Caron"
#> [67] "Can a ‘Cortisol Cocktail’ Help Relieve Stress?"
#> [68] "Influencers say that the concoction can boost energy and help you relax. We asked experts whether that’s true."
#> [69] " By Caroline Hopkins Legaspi"
#> [70] "Measles Cases Hit Highest Total Since U.S. Eliminated the Disease"
#> [71] "Experts worry that if vaccination rates do not improve, deadly outbreaks will become the new normal."
#> [72] " By Teddy Rosenbluth and Jonathan Corum"
#> [73] "An analysis by the American Association for the Advancement of Science shows the impact of the administration’s budget plan on the kind of studies that produce the most breakthroughs."
#> [74] "By William J. Broad"
#> [75] "The N.I.H. has terminated hundreds of diversity grants awarded to young researchers, many of whom come from the very places that supported Trump."
#> [76] "By Kate Zernike"
#> [77] "Susan Burton, reporter and host of the podcast “The Retrievals,” talks about the alarming number of patients who report feeling significant pain during their C-sections."
#> [78] "By Susan Burton, Laura Bult, Pedro Piccinini, Laura Salaberry and David Seekamp"
#> [79] "Susan Burton, reporter and host of the podcast “The Retrievals,” talks about the alarming number of patients who report feeling significant pain during their C-sections."
#> [80] "By Susan Burton, Laura Bult, Pedro Piccinini, Laura Salaberry and David Seekamp"
#> [81] "The regulatory agency confronts a future determined by a health secretary hostile to its mission."
#> [82] "By Jeneen Interlandi"
#> [83] "How the new health secretary, Robert F. Kennedy Jr., is dismantling the agency."
#> [84] "By Jeneen Interlandi"
#> [85] "The health secretary has used peer pressure to persuade food makers to nix synthetic dyes. The candy industry is holding out, arguing American consumers like bright sweets."
#> [86] "By Sheryl Gay Stolberg and Julie Creswell"
#> [87] "President Trump’s domestic policy law jeopardizes plans to reopen one rural county’s hospital — and health coverage for hundreds of thousands of state residents."
#> [88] "By Eduardo Medina"
#> [89] "Despite the unspeakable horror of her youth, she embraced a school of psychotherapy that stresses empathy and the belief that everyone can change for the better."
#> [90] "By Trip Gabriel"
#> [91] "He turned a small family business into a billion-dollar weight-loss empire by replacing calorie counting and forbidden foods with “just add milk.”"
#> [92] "By Robert D. Hershey Jr."
#> [93] "Advertisement"
#> [94] "Advertisement"