Automating clicks on webpage

[Luc]

Not many people around here use Python as a programming language, which is ok.

Maybe I need to attack this differently.

To find something on Google, you need to formulate your question with the right keywords and you will probably hit what you are looking for. As for now, I am not finding what I'm looking for.

Let me explain on what I want to do.

I want to run on-demand (not always, not automatically) a script that will perform a few clicks on a webpage.

I tried Python with Selenium and the issue is that myfitnesspal.com sees me as a Robot when I run the script using either Firefox or Chrome. I tried to click the human box but it keeps looping. I cannot seem to be able to go around that.

I figured out the queries that I wanted to run in Javascript and added .click() in the developer console. That works but it's 1 at the time. I cannot run the 3 queries on the page as the page is reloaded after each .click(). So, copy-paste query 1, run, wait, copy-paste query 2, run, wait, and then query 3.

There must be a way or widget, or app or something else that I can input those commands in a loop or not so that it runs command 1, reload, command 2, reload, and command 3.

I tried snippets and I have the same issue as the console. The only plus with the snippet is that I can do Command1, Command2, and Command 3 and run them one after another. However, that's not automated to me.

So, since Selenium sees me as a robot and I cannot seem to be able to disable that, what else is out there? I don't need to enter data, just click on specific links on a website. Does someone use something that allows you to navigate multiple pages with automated clicks?

 

[House Boy]

Wish i knew what you're talking about but I just use Siri.

Hey SIRI....get me out of this post

 

[Luc]

My goal is to avoid using my mouse to click on web pages. I want to run a script that will navigate the web and click for me automatically.

I know this is a shaving forum and not a programmer's forum or IT forum.

 

[Toothpick]

I've been using ChatGPT a lot more often lately. I find it easier than google to come up with some things. IDK if this will help, but you might DL chatGPT with your phone or tablet and try using that to make a script.



Automating website clicks can be done using libraries like `selenium`, which interacts with web browsers to simulate user actions like clicking buttons, filling forms, and navigating through web pages.

Here's a basic Python script using Selenium for automating clicks on a button:

```python
# Install Selenium: pip install selenium
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from webdriver_manager.chrome import ChromeDriverManager

# Setup Chrome options (you can use this to run the browser in headless mode if needed)
chrome_options = Options()
chrome_options.add_argument("--start-maximized") # Opens the browser in full screen
# Uncomment the following line to run Chrome in headless mode
# chrome_options.add_argument("--headless")

# Initialize the Chrome webdriver
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=chrome_options)

# Open the website
driver.get("Example Domain - https://example.com") # Replace with your desired website

# Find the button by its element (change this to match the element's locator on the site)
# For example, locating the button by its ID:
button = driver.find_element(By.ID, "button-id") # Replace with the button's actual ID

# Click the button
button.click()

# Wait for a few seconds to observe the result (optional)
import time
time.sleep(5)

# Close the browser
driver.quit()
```

### Key components:
- **Selenium**: Controls the browser for automation.
- **Chrome WebDriver**: A browser driver that Selenium uses to interact with Chrome. This can be managed automatically with `webdriver_manager`.
- **Locating elements**: The `find_element` method can locate elements on the page using different strategies (`By.ID`, `By.XPATH`, `By.NAME`, etc.).

### Setup instructions:
1. Install Selenium and WebDriver manager:
```bash
pip install selenium webdriver-manager
```
2. Replace `Example Domain - https://example.com` with the URL of the site you want to automate.
3. Modify the `find_element` method according to the structure of the webpage you're automating. For example, you might need to locate the button by `class name`, `XPath`, or `CSS selector`.

Would you like me to modify the script for a specific website or action?

 

[Toothpick]

you can tell it to modify the script for a specific website or action too.

 

[Toothpick]

To automate clicks on MyFitnessPal.com using Selenium, we'll need to identify the elements that correspond to the actions you'd like to perform (e.g., logging in, navigating through the site, etc.). Below is a basic script that logs into MyFitnessPal and simulates some actions (such as clicking around the dashboard or other sections).

### Python Script for Automating Actions on MyFitnessPal.com

```python
# Install Selenium: pip install selenium
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from webdriver_manager.chrome import ChromeDriverManager

# Setup Chrome options (headless can be used if no GUI is required)
chrome_options = Options()
chrome_options.add_argument("--start-maximized") # Opens the browser in full screen
# chrome_options.add_argument("--headless") # Uncomment for headless mode

# Initialize the Chrome WebDriver
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=chrome_options)

# Open MyFitnessPal's login page
driver.get("https://www.myfitnesspal.com/account/login")

# Find the username and password input fields and enter login credentials
username = driver.find_element(By.ID, "username") # Adjust based on actual login field ID
password = driver.find_element(By.ID, "password") # Adjust based on actual password field ID

# Input your login credentials (replace with your own or load from a secure source)
username.send_keys("your_username") # Replace with your username
password.send_keys("your_password") # Replace with your password

# Find the login button and click it
login_button = driver.find_element(By.XPATH, '//button[@type="submit"]') # Adjust if necessary
login_button.click()

# Wait for a few seconds to allow the page to load
import time
time.sleep(5)

# Now that you're logged in, you can automate other clicks.
# For example, let's click on the "Food" tab in the dashboard.
food_tab = driver.find_element(By.LINK_TEXT, "Food") # Adjust this locator if needed
food_tab.click()

# Wait for a few seconds to observe
time.sleep(5)

# Close the browser after the task is complete
driver.quit()
```

### Steps to run the script:
1. **Install the necessary packages**:
```bash
pip install selenium webdriver-manager
```
2. **Modify the script** with your MyFitnessPal credentials:
- Replace `"your_username"` and `"your_password"` with your MyFitnessPal credentials (you can store them securely and retrieve them using a password manager or environment variables).

3. **Customize the actions**:
- I've included an example for clicking the "Food" tab after login, but you can expand this by locating other elements like buttons, forms, or links to automate further actions (e.g., logging food, checking progress).

4. **Element locators**:
- Ensure that you inspect the webpage to confirm the correct `ID`, `XPATH`, or other selectors for elements you want to click. You can inspect elements using browser dev tools (right-click -> inspect).

Would you like to automate additional actions on MyFitnessPal, such as logging meals or workouts?

 

[Toothpick]

Here I told ChatGPT "it thinks I'm a robot, how do I fix that?"




When automating interactions with websites like MyFitnessPal, they may use CAPTCHA or other bot detection techniques to prevent automated logins. Here's how you can potentially work around these issues:

### Approaches to Handle CAPTCHA or "Robot" Detection:

1. **Use Human-Assisted Login:**
- One way to bypass CAPTCHA is to manually log in once and let Selenium handle further automation after the CAPTCHA. After you log in, Selenium can take over to automate further interactions. You can store session cookies to avoid repeated logins (see cookie handling below).

2. **Use a Browser Profile (Avoiding CAPTCHA on Repeat Visits):**
- By using a browser profile, Selenium can maintain a persistent session (e.g., cookies and cache), so you don't need to log in each time. This may help avoid CAPTCHA on subsequent visits.

```python
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager

# Path to your Chrome profile
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("user-data-dir=./chrome-profile") # Create a folder to store your profile

# Initialize the Chrome WebDriver with the profile
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=chrome_options)

# Open MyFitnessPal (you may need to log in once manually)
driver.get("https://www.myfitnesspal.com/account/login")

# Selenium will now use the same profile with cookies and session saved
```

3. **Use Two-Step Automation (Human Input for CAPTCHA):**
- You can pause the script at the CAPTCHA step and allow human interaction to solve it, then continue the automation after successful login.

```python
driver.get("Free Calorie Counter, Diet & Exercise Journal | MyFitnessPal - https://www.myfitnesspal.com/account/login")

# Pause here to allow the user to manually solve CAPTCHA
input("Please solve the CAPTCHA, then press Enter to continue...")

# Continue with automation after the user solves CAPTCHA
food_tab = driver.find_element(By.LINK_TEXT, "Food")
food_tab.click()
```

4. **Browser Automation Tools (Undetected WebDriver):**
- Many websites detect the Selenium WebDriver by looking for browser automation flags. You can use libraries like **undetected-chromedriver**, which attempt to make Selenium less detectable.

```bash
pip install undetected-chromedriver
```

Here’s how to integrate `undetected-chromedriver` into your script:

```python
import undetected_chromedriver as uc

# Use undetected-chromedriver to avoid detection
driver = uc.Chrome()

# Open MyFitnessPal
driver.get("Free Calorie Counter, Diet & Exercise Journal | MyFitnessPal - https://www.myfitnesspal.com/account/login")

# Perform login and further actions as usual
```

5. **Use API Instead of UI Automation:**
- If MyFitnessPal provides an API, it's better to interact with it directly rather than automating the UI. This will bypass CAPTCHAs and robot detection altogether. Unfortunately, MyFitnessPal has removed public access to its API, but some third-party APIs exist that allow basic functionalities such as logging

 

[Toothpick]

I tried Python with Selenium and the issue is that myfitnesspal.com sees me as a Robot when I run the script using either Firefox or Chrome. I tried to click the human box but it keeps looping. I cannot seem to be able to go around that.

I also put this quote in ChatGPT and got this info...



If MyFitnessPal keeps detecting you as a bot when using Selenium and looping CAPTCHA, this is likely due to how Selenium interacts with the browser. Many modern websites, including MyFitnessPal, have sophisticated bot-detection techniques that can identify automated browser interactions, such as using Selenium. This is often done by looking for specific browser signatures or behaviors commonly associated with automation tools.

Here are a few alternative approaches to deal with this issue:

### 1. **Use Undetected WebDriver (`undetected-chromedriver`)**
Websites may detect that you are using a WebDriver, which is why `undetected-chromedriver` is an alternative that makes the browser appear as though it's not automated. This often bypasses CAPTCHAs and bot-detection systems.

```bash
pip install undetected-chromedriver
```

Here's an example script using `undetected-chromedriver`:

```python
import undetected_chromedriver as uc

# Initialize undetected-chromedriver
driver = uc.Chrome()

# Open MyFitnessPal login page
driver.get("https://www.myfitnesspal.com/account/login")

# Find the login fields and button, then log in
username = driver.find_element(By.ID, "username")
password = driver.find_element(By.ID, "password")

username.send_keys("your_username")
password.send_keys("your_password")

login_button = driver.find_element(By.XPATH, '//button[@type="submit"]')
login_button.click()

# Proceed with additional actions like navigating to the food log
```

### 2. **Use Browser Profiles**
Using a browser profile can help you avoid triggering bot-detection mechanisms. This technique saves cookies and session data, meaning you won't have to log in repeatedly, and it can bypass CAPTCHA after an initial login.

```python
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager

# Use an existing Chrome profile
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("user-data-dir=./chrome-profile")

# Initialize the driver with the profile
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=chrome_options)

# Open MyFitnessPal
driver.get("Calorie Tracker & BMR Calculator to Reach Your Goals | MyFitnessPal - https://www.myfitnesspal.com")

# If logged in already, you should be able to continue interacting
```

Steps:
- Log in to MyFitnessPal manually with your profile.
- Selenium will use that session, skipping CAPTCHA in future script runs.

### 3. **Use Puppeteer with Python (Through Pyppeteer)**
Puppeteer is another browser automation tool that's harder for websites to detect compared to Selenium. Pyppeteer is a Python wrapper for Puppeteer.

```bash
pip install pyppeteer
```

Example usage:

```python
import asyncio
from pyppeteer import launch

async def main():
browser = await launch(headless=False)
page = await browser.newPage()

# Go to MyFitnessPal login page
await page.goto("Free Calorie Counter, Diet & Exercise Journal | MyFitnessPal - https://www.myfitnesspal.com/account/login")

 

[Toothpick]

I don't think it copy/pasted the entire thread. It seems as though some info from the ChatGPT response is missing from my copy/paste posts above. But it's free to download if you think it will help.

 

[Luc]

@Toothpick, I will give a second go at the selenium thing. The only bit that I did not try was the template that you posted regarding the undetected driver. The rest works until the page is refreshed. So only half the script executes.

I got a few ideas with Tampermonkey. I will report back if I get something that works.

 

[Lightcs1776]

I'm learning python as it can be used to manage network devices (I work as a network engineer), and I also started teaching an undergraduate course in databases. NoSQL databases require a programming language and Python fits the requirements quite well. Let us know how your script works out for you. I love seeing different methods of accomplishing a task.

 

[StillShaving]

I was curious, are you using their REST interface or scraping output from their web or App interface.

If something was missing in their interface, maybe they would be open to adding it to benefit all users. Requesting an enhancement and waiting on them to an answer is not as interesting as figuring out how to use what they already provide. But from what I recall you were looking for something that users in general would probably want........ some method to anticipate meals and/or quickly copy them from recent entrees into new ones.

 

[Luc]

I'm learning python as it can be used to manage network devices (I work as a network engineer), and I also started teaching an undergraduate course in databases. NoSQL databases require a programming language and Python fits the requirements quite well. Let us know how your script works out for you. I love seeing different methods of accomplishing a task.

The Python script works for me, to an extent. I'm still working on the ChatGPT posts that Jason provided.

I was curious, are you using their REST interface or scraping output from their web or App interface.

If something was missing in their interface, maybe they would be open to adding it to benefit all users. Requesting an enhancement and waiting on them to an answer is not as interesting as figuring out how to use what they already provide. But from what I recall you were looking for something that users in general would probably want........ some method to anticipate meals and/or quickly copy them from recent entrees into new ones.

I use the web interface, so I try running everything on my desktop using Chromium, Chrome, or Firefox. I posted on the MFP forum to ask for a feature and got 1 upvote so I don't think it will get anywhere.

The API website mentions: We are not accepting requests for API access at this time.

The information/features that I need are on the web interface. Basically what I want to do is :
-Go in the food diary
-Check if I have any calories logged in the breakfast section.
--If there are calories, skip to the next day
--If there are no calories, use the copy meal from the date link and pick the last item (the last item equals the same day. So copy last Sunday to this Sunday).
-Done, skip to the next day.

I would start with 1 day and eventually put a loop to do 2 weeks, 4 weeks, or something.

With the ChatGPT scripts, I need to re-login even if I'm already logged in. There's the captcha thing, and then, I could probably look at scrolling the elements to pick the calories logged.

However, I'm stuck at the login screen + captcha. I tried running Chromium and the page opens but I can't log in automatically. I tried manually but the script doesn't pause either.

This could be a Chromium issue as I was not able to use Tampermonkey with Chromium to have a Hello World printed on google.com.

I switched Tampermonkey to Firefox and bam, it works on Google. So I'm reworking TamperMonkey with Firefox. Now I'm having issues picking the breakfast calories as it doesn't have a class or ID. So I need to figure out a way to scroll through the bottom part of the breakfast meal and pick the first item (which is the calories) or something.

I tried installing a Tampermonkey script that was adding the Weight Watcher points so that I could recycle the code but the script I tried didn't work.

I will need to reinstall Chrome or maybe try FireFox with Selenium/Python. I don't know if it's recent but Chrome is being updated and patched very often compared to FireFox/Chromium. Each update can potentially disturb the webdriver for Selenium.

Anyways, that's where I'm at.

 

[Toothpick]

No idea what you are talking bout. I just like playing with ChatGPT for this stuff.

Here is its answer to "a python tamper monkey script that adds weight watcher points" . It wants to use JavaScript since it says Tampermonkey runs java in the browser.

// ==UserScript==
// @Name Weight Watchers Points Adder
// @namespace Home | Tampermonkey - http://tampermonkey.net/
// @version 1.0
// @description Automatically adds Weight Watchers points
// @author Your Name
// @match *://*/* // Adjust this to the specific page you want to target
// @Grant none
// ==/UserScript==

(function() {
'use strict';

// Your code here
const addPoints = (points) => {
// Find the input field where points need to be added
const inputField = document.querySelector('#inputFieldSelector'); // Change to the actual selector
if (inputField) {
inputField.value = points; // Set the points
// If there is a submit button, you can click it programmatically
const submitButton = document.querySelector('#submitButtonSelector'); // Change to the actual selector
if (submitButton) {
submitButton.click(); // Submit the form
}
}
};

// Example usage
const pointsToAdd = 10; // Set the points you want to add
addPoints(pointsToAdd);
})();

Steps to Customize:​

1. Adjust the Match URL: Change the @match field to the specific URL or domain where you want the script to run.
2. Select Input Field and Button: Update the #inputFieldSelector and #submitButtonSelector with the actual selectors for the input field and button on the Weight Watchers page.
3. Points Logic: Modify the logic to determine how many points to add, based on your needs.

Testing Your Script:​

1. Install Tampermonkey in your browser.
2. Create a new script and paste the template.
3. Test it on the Weight Watchers site to ensure it works as intended.

Let me know if you need any specific adjustments or additional features!


ChatGPT is also so much better in a computer browser. Here's a screenshot. You can probably just make it write you own food tracking app.

 

[maxman]

25 years as a tech and I can only do PowerShell. Not sure if that's sad or impressive.
Feels kinda sad. lol.

 

[Luc]

Thanks @Toothpick, I will give that a go. It's mainly a syntax thing that I need to figure out (i.e. fill the blanks to grab the right field). That helps for sure.

@maxman, I know what you mean. I can do a bunch of other things but web automation is outside my comfort zone. That's why I decided to dive in.

 

[DrStrange]

Not many people around here use Python as a programming language, which is ok.

Maybe I need to attack this differently.

To find something on Google, you need to formulate your question with the right keywords and you will probably hit what you are looking for. As for now, I am not finding what I'm looking for.

Let me explain on what I want to do.

I want to run on-demand (not always, not automatically) a script that will perform a few clicks on a webpage.

I tried Python with Selenium and the issue is that myfitnesspal.com sees me as a Robot when I run the script using either Firefox or Chrome. I tried to click the human box but it keeps looping. I cannot seem to be able to go around that.

I figured out the queries that I wanted to run in Javascript and added .click() in the developer console. That works but it's 1 at the time. I cannot run the 3 queries on the page as the page is reloaded after each .click(). So, copy-paste query 1, run, wait, copy-paste query 2, run, wait, and then query 3.

There must be a way or widget, or app or something else that I can input those commands in a loop or not so that it runs command 1, reload, command 2, reload, and command 3.

I tried snippets and I have the same issue as the console. The only plus with the snippet is that I can do Command1, Command2, and Command 3 and run them one after another. However, that's not automated to me.

So, since Selenium sees me as a robot and I cannot seem to be able to disable that, what else is out there? I don't need to enter data, just click on specific links on a website. Does someone use something that allows you to navigate multiple pages with automated clicks?

I understand Selenium's point of view.

If automating clicks on a website isn't a robot,
then what you do think "robot" means in this context?

 

[Luc]

I understand Selenium's point of view.

If automating clicks on a website isn't a robot,
then what you do think "robot" means in this context?

The whole concept of Robot is fine. I'm just trying to avoid a few clicks by automating them. If the robot concept of Selenium is blocked on the website where I try to do that, that's ok. There's probably another way that I don't know. Some software somewhere might allow it. Maybe it's simply by coding an HTML file a certain way. I won't know if I don't ask.

Right now, my manual robot task works when I use a snippet with 1 line of code that I named
Click-1
Click-2
Click-3

I open the right webpage and I run the snippets one after another. Technically, it's manual but I still save 3 clicks per snippet. If I can do that, I'm sure there's a way to make the 3 clicks happen in one shot instead of clicking my 3 snippets manually. Maybe I need to put a delay in between but I'm not sure I can do that with a snippet...

Let me put it this way then. Do you you known another way that would work?

 

[StillShaving]

I have not used python very much, but I might expect it to be easier to run a PY script from the command line. Using its REST library to help establish a connection and retrieving output (and posting back new changes/updates). I think there is a library that breaks the JSON down into key/value pairs for consumption. Or perhaps could try and use a more basic websocket like approach where your program scans/parses through the whole output looking for the appropriate keyword and using that to build a new string buffer to send back over the connection to effect an update.

MFP API - MyFitnessPal Developer - Diary - GET

myfitnesspalapi.com

But I could misunderstand what you are wanting to do or what their API offers.

 

[Luc]

Their API mentions no new connections. I’ve seen people posting that they’ve been waiting for over 1 year and still waiting. So I don’t thing the API is a viable option.

I got a few things that I can try with Tampermonkey. Selenium IDE is different from what I’ve heard than Selenium (full package).

I will keep digging.